In Episode 281, we introduced Microsoft OneLake with a high-level overview. Now we're going deeper with a discussion on the Parquet format, why Microsoft went with the Delta Lake variation, and what Delta Lake format brings to the table (no pun intended). We'll also examine some "behind the scenes" aspects of file management, and why you'll still be using the GUI to create most of your objects.
Onelake is Microsoft's solution to the demand for centralizing all data in one location, eliminating the need to transfer it across multiple systems. We expect this to play out further however, when we consider scenarios like data sovereignty, geographical data distribution, separation of subsidiary data, and even departmental budgets that may necessitate multiple instances of OneLake.
We round out our OneLake deep dive with a conversation on the Direct Lake Mode option for importing data into Power BI and Eugene shares his perspective on why everyone may not be rushing to jump on the bandwagon just yet.
We hope you enjoyed this deep dive into Microsoft OneLake! If you have questions or comments, please send them our way. We would love to answer your questions on a future episode. Leave us a comment and some love ❤️ on LinkedIn, X, Facebook, or Instagram.
The show notes for today's episode can be found at Episode 282: OneLake - A Deep Dive. Have fun on the SQL Trail!
As you start using Fabric, having a central location for your data is crucial. OneLake acts as this unified destination, offering a single, consolidated repository for all your data. In this podcast episode, we explore the core features of OneLake and its benefits with our guest, Mariano Kovo, and discuss how it efficiently handles large amounts of data from diverse sources. We'll also dive into the importance of how your data is presented to Azure services, focusing on the Delta Parquet format.
Did you know you can explore OneLake data directly through Windows Explorer? Microsoft aims to make a single copy of your data accessible across multiple services, eliminating the need for constant data movement. Shortcuts make it easier to access your data seamlessly within the OneLake environment, enhancing efficiency and accessibility.
We hope you enjoyed this foundational episode on Microsoft OneLake! If you have questions or comments, please send them our way. We would love to answer your questions on a future episode. Leave us a comment and some love ❤️ on LinkedIn, X, Facebook, or Instagram.
The show notes for today's episode can be found at Episode 281: OneLake - The OneDrive for Data. Have fun on the SQL Trail!
At the Microsoft Build Conference in May 2023, Microsoft announced the new Fabric, where you could slice and dice all your data harmoniously within the environment. A few months later, Kevin, Eugene, and I discussed this evolution of the Azure Data platform in episode 267, and our thoughts on the vision for it's future, our expectations, and predictions.
Now, more than a year later, we decided it's a good time to take an in-depth look at the platform to see what goals have come to fruition, what predictions have come true, and what may have changed. In this introduction to Season 8, we'll get the conversation started.
In the next 10 episodes we'll be taking a deep dive into the reality of what Microsoft Fabric is today, navigating through the nuances, complexities, and sheer vastness of the product. We'll break it down into digestible chunks focused on specific aspects such as:
If you have questions or comments about Microsoft Fabric, please send them our way. We would love to answer your questions on a future episode. Leave us a comment and some love ❤️ on LinkedIn, X, Facebook, or Instagram.
The show notes for today's episode can be found at Episode 280: A Focus on Microsoft Fabric. Have fun on the SQL Trail!
If you use SQL Server, you will eventually have to migrate that instance somewhere – to a new version, a new server, the cloud . . . somewhere.
Or perhaps you'll find yourself migrating from another database into SQL Server.
No matter which way you slice it, SQL Server migrations can be daunting, not to mention complex and time-consuming. While we know there are risks and many things that can go wrong, the "new" Microsoft continues to put time and effort towards making successful SQL Server migrations attainable for everyone.
In this episode of the podcast, we chat with Tejas Shah and Sudhir Raparla, 2 of the Microsoft Project Managers responsible for SQL Server migration tooling. They share practical perspectives on approaching your SQL migration with confidence and the tools and enhancements that will help.
During the conversation, Tejas and Sudhir also take us through the 5 migration steps they want you to consider as you undertake your SQL Server migration process.
Even though we’ve migrated thousands of databases, I had to go back and peek at a couple of the new features the migration tooling team has added. One intriguing addition is the Azure SQL Pricing repository, which is part of the SQL Server Migration assistant and can help with determining costs based on industry standards, deployment recommendations, target sizing, and monthly savings based on your unique scenario.
Let us know what you think! What SQL migration features have you come to trust and rely on? Did you get any good takeaways from today's podcast or have some questions? Leave us a comment and some love ❤️ on LinkedIn, X, Facebook, or Instagram.
The show notes for today's episode can be found at Episode 279: SQL Server Migrations Demystified. Have fun on the SQL Trail!
Can you run SQL Server on Azure VM? Which VM is best? Is running SQL Server on a VM in Azure the right choice? Find out in this insightful episode with Anders Pedersen!
With over 10 different SQL Server services now offered in Microsoft Azure, it can be difficult to know how you want to run your environment. Sometimes, the old ways are the best ways for an organization, and running SQL Server on a VM in Azure is the right fit.
In this episode of the SQL Data Partners Podcast, we chat with Anders Pedersen about his experience moving their systems to Azure VMs. We discuss some of the tiering issues, the newest storage tier being rolled out, and how he manages upgrades.
Join us for another informative podcast where a seasoned database administrator shares their experience of managing a SQL Server environment.
Did you get any good take-aways from today's podcast or have some questions? Leave us a comment and some love ❤️ on LinkedIn, X, Facebook, or Instagram.
The show notes for today's episode can be found at Episode 278: Running SQL Server on Azure VMs. Have fun on the SQL Trail!
Is testing out pgAdmin on your to-do list?
In this episode of the podcast, we chat with Ryan Booz, a PostgreSQL advocate at Redgate, about how a SQL Server professional might begin a dive into PostgreSQL, one of the most popular open source databases in the world.
Ryan came from a career background in SQL Server, but after experiencing his accidental "jump-into-the-deep-end" PostgreSQL moment, he hasn’t looked back.
Naturally, open source presents DBAs and their organizations with many desirable features, but there are certain drawbacks as well. Ryan shares how he navigated his transition into PostgreSQL and raises some points to consider if you are thinking about a switch. We discuss a few of the land mines you might encounter along the way as well as terminology differences in this space.
Be sure to check out Planet PostgreSQL for the most recent blog posts from the very folks that are contributing code to the PostgreSQL project.
Have you shifted from SQL Server to PostgreSQL? How'd it go? Did you get any good take-aways from today's podcast or have some questions? Leave us a comment and some love ❤️ on LinkedIn, X, Facebook, or Instagram.
The show notes for today's episode can be found at Episode 277: PostgreSQL for the SQL Server Crowd. Have fun on the SQL Trail!
Listener beware! This episode is full of danger as we tackle an interesting use case for Dynamic SQL. Dynamic SQL generally has a bad reputation in SQL Server circles, and with good reason. Dynamic SQL can open the door to many undesirable results - SQL Injection attacks being the most frightening of these. It can also be difficult to read, making maintenance problematic; however, in this episode one brave soul - Marathon's own Laura Moss - explains how she uses Dynamic SQL to help refresh a subset of production data to be used in their development environments. You know we are always suckers for an interesting use case and Laura delivers big time. While you won’t be able to plug and play her example into your environment, we hope it gets the wheels turning if you struggle to update your test environments.
Have you found a way to use Dynamic SQL as a tool for good and not evil? Did you get any good take-aways from today's podcast or have some questions? Leave us a comment and some love ❤️ on LinkedIn, Twitter/X, Facebook, or Instagram.
The show notes for today's episode can be found at Episode 276: Dynamic SQL and Testing in Isolation. Have fun on the SQL Trail!
Do you find yourself repeating the same actions when pulling SQL Server performance metrics?
Performance tuning a troublesome SQL Server can be a challenge. Luckily the community continues to produce wonderful folks like Erik Darling who contribute their knowledge to make your life a bit easier. In this episode of the SQL Data Partners podcast, we sit down with Erik and discuss the scripts he built to gather performance metrics. While every potential issue is not captured in these scripts, they'll help you start gathering information so you can decide on the next step to take.
Have you used Erik’s scripts before? Let us know!
The show notes for today's episode can be found at Episode 272: Performance Tuning Scripts. Have fun on the SQL Trail!
We're kicking off Season 7 with containers!
Spinning up a VM may not be such a big deal anymore; however, most of us still have to request from another group one and wait. Even waiting on an Azure VM can be somewhat painful. Wouldn't it be nice to forget about setting up another development environment just to test something that isn't going to stick around?
Our guest today is Chuck Bryan, and he talks to us about how he is using containers to support his environments and the flexibility it provides to him in his development.
While the Linux containers used to get lots of love, there haven't been too many feature updates lately as much of the focus is on azure services. What is cool to me is there are tools out there that can help us folks running windows get up and running without having to wait on our infrastructure to upgrade to Windows server 2016--or have Azure spend.
Chuck gives us some insights on how he got started with containers. We discuss what environments might benefit from them--and which ones won't. He also gives us a couple of tips on the best places to get started.
The show notes for today's episode can be found at Episode 266: Working With Containers. Have fun on the SQL Trail!
Compañeros! In May, SQL Data Partners was acquired by Marathon Consulting. There were many moving parts to get this done and we are excited to finally bring this news to the podcast. In this episode we introduce the principles of Marathon Consulting, and we discuss the future of the podcast. Make sure your speakers are adjusted properly when you listen to this episode. If you have any questions or comments, please let us know via LinkedIn—we always look forward to hearing from you.
The show notes for today's episode can be found at Episode 265: WHY and What's Next. Have fun on the SQL Trail!
A backup is only helpful when you can restore the backup. This should not prove to be overly shocking; however, normally we get the satisfaction of taking a backup and feel like all is well. Ideally, we should be testing our backups on some cadence—and it should be more frequent than the annual disaster recovery test. In this episode of the SQL Data Partners Podcast we talk with Chris Marshall, owner of Verified Backups, about how he has solved this problem and how it can benefit your organization. We discuss some of the requirements you’ll need to address in your own testing, and we give some parameters for who might be a good candidate for the service Chris and his team provide.
The show notes and video for today's episode can be found at https://sqldatapartners.com/2023/04/12/episode-264-verified-backups. Have fun on the SQL Trail!
When we couldn’t confirm a guest speaker by recording time, Eugene, Kevin, Amanda, and I decided the show must go on, so we went into the ideas vault and pulled out a couple of listener questions we hadn’t answered. In this episode, we talk about how to create your own documentation, SSMS installs, SOS_Scheduler_Yield and Eugene gives us an update on SQL Bits and the conference crud he picked up along the way.
The show notes and video for today's episode can be found at https://sqldatapartners.com/2023/03/29/episode-263-sql-server-this-n-that. Have fun on the SQL Trail!
Learning new technology can be difficult—especially when it’s a technology in an area you are not familiar with. In this episode, Eugene walks us through some of the challenges he has faced getting started with Azure Synapse. We hope that this episode might be helpful to those trying to learn new features.
The show notes and video for today's episode can be found at https://sqldatapartners.com/2023/03/15/episode-262-learning-azure-synapse-pt-1. Have fun on the SQL Trail!
It would appear Microsoft has run out of words to use for new features/products, so they are just going to recycle existing words for a bit of fun at our expense. In this episode we explore the Azure Data Lakehouse—which is not a data lake, nor a traditional data warehouse and Books Online (BOL) refers to it as Databricks Lakehouse. Luke Moloney walks us through how combining the flexibility of data lake storage with some ACID transaction and data governance gives organizations looking to analyze their data a new option. The hope is the cost to build, store, and analyze data will be easier and more approachable for organizations who don’t want to go with the traditional data warehouse model. This still feels like an Enterprise feature to me but let me know if your organization would be interested in this approach. As always, special thanks to Luke and the folks at Microsoft for making themselves available to us.
The show notes and video for today's episode can be found at https://sqldatapartners.com/2023/03/01/episode-261-the-data-lakehouse. Have fun on the SQL Trail!
We’re back with another SQL Server 2022 feature—Contained Availability Groups. Have you ever failed over to a secondary node and then found a job or login missing? This new Enterprise-only feature aims to solve this problem by ‘containing’ all the objects—master, msdb, AND the availability database all in one group so you can manage everything you need when you have to fail over. Brandon Leach returns after more than 200 episodes to give us the lowdown on this new feature and walk us through where you should start planning your upgrade.
The show notes and video for today's episode can be found at https://sqldatapartners.com/2023/02/15/episode-260-contained-availability-groups. Have fun on the SQL Trail!
We kick off the new year discussing a new feature in SQL Server 2022 called Transact-SQL snapshot backups. Good luck with the Google search on this feature--we know we had fun. 😊 These are not the snapshot backups you were frustrated with in 2005. We talk a little bit about who will be interested in this feature and what scenarios it will come in handy. The hardware vendors seem particularly excited about this future as they can now introduce snapshots without needing a third-party application like Veeam to do that for them. We predict more collaboration between the database administrator and the hardware administrator with this feature. We also get a couple of use case scenarios from the community that we share in this episode. Are you looking forward to using snapshots more effectively in your environment? Please let us know your thoughts!
The show notes and video for this episode can be found at https://sqldatapartners.com/2023/01/18/episode-259-t-sql-snapshot-backups/. Have fun on the SQL Trail!
As we close out the year, we take a look back and think about all those SQL Server 2022 episodes we didn’t record this year—and then make the goal to increase this count in the new year. 😊 We discuss some of the Cardinality Estimation changes in this version and the scenarios in which this may affect other features. I ask Kevin and Eugene what technologies they will be focusing on in the year ahead and how many books Kevin is going to try and write. From all of us at SQL Data Partners, we wish you and yours a Happy New Year!
The show notes and video for today's episode can be found at https://sqldatapartners.com/2022/12/28/episode-258-leave-2022-enter-2023. Have fun on the SQL Trail!
Have you ever thought about working for yourself? It can be very enticing for some, and in this episode, we talk with John Sterrett about the pros and cons of running your own organization. We tackle some of the reasons you might get started and we share some of our experiences of working in our own organizations. Each person’s journey is going to be different and as hard as it will be, you need to ensure you don’t allow others success or failure to overly influence your decisions. As the work environment has shifted in the last two years, it may that be some of the previous ‘whys’ no longer apply. If you have thoughts around this, let us know!!
The show notes and video for today's episode can be found at https://sqldatapartners.com/2022/11/30/episode-257-the-business-of-working-for-yourself. Have fun on the SQL Trail!
Seven years ago, I started the SQL Data Partners Podcast. We have now arrived at Episode 256, and we dedicate this episode to you, the community. Thanks to everyone who has joined us along the way, and we hope to continue providing content you can enjoy on the go while still connecting with your SQL Server community. While we originally thought we might focus on SQL Server 2022, we really found there are just lots of great topics to be covering in the future. We hope you enjoy this special community edition of the SQL Data Partners Podcast.
The show notes and video for this episode can be found at https://sqldatapartners.com/2022/11/03/episode-256-the-community-episode/. Have fun on the SQL Trail!
One of the biggest challenges with developers using object-relational mapping (ORM) tools is the bad queries or procedures they create. Sure, most tools can create CRUD (Create, Read, Update, Delete, Upsert) procedures, but what about the extremely advanced safe dynamic Search stored procedures? My bet is they are super inefficient and could be improved. In this episode of the SQL Data Partners Podcast, we talk with Kevin Martin, the creator of sp_CRUDGen, about how his procedure can help your developers create better CRUD-type stored procedures. As always, we are super excited about sharing what our great community is working on, and we think there will be value in the work Kevin has put together. Have you used sp_CRUDGen? We’d love to get your thoughts.
The show notes and video for today's episode can be found at https://sqldatapartners.com/2022/10/05/episode-255-automate-complex-crud-with-sp_crudgen/. Have fun on the SQL Trail!
Would you be interested in a tool to help you do code review checks and provide developers guidance on best practices? In this episode of the SQL Data Partners Podcast, we talk with Kevin Martin, the creator of sp_Develop, about how his procedure can help you review naming conventions, table conventions, data type conventions, SQL code conventions, data issues, and even configuration issues. Kevin was approached about writing a best practice document but ended up writing a stored procedure you and I can use to help our environment. We love sharing what community members have done and hope you enjoy this conversation with Kevin on the tool. He is looking for folks to help him build out items, so if you have a suggestion, be sure to include it in GitHub.
The show notes and video for today's episode can be found at https://sqldatapartners.com/2022/09/21/episode-254-code-smell-checks-with-sp_develop/. Have fun on the SQL Trail!
In this episode, we dive into the Power Apps offering and discuss the pros and cons of using this latest Microsoft offering. Some definitions are given and hopefully this episode can orient folks on how to position Power Apps in their organizations.
The show notes and video for today's episode can be found at https://sqldatapartners.com/2022/09/07/episode-253-go-mobile-with-power-apps. Have fun on the SQL Trail!
Knowing your SQL Server is patched when an issue occurs is worth the effort of keeping it patched. While I don’t suggest a zero-day install for most folks, setting up a regular cadence to check is a reasonable ask if you have to manage a system. In this episode of the SQL Data Partners Podcast, we discuss the need for patching and how you might go about it. Special shoutout to Episode 112 with the late Robert Davis on the same subject. We would also like to thank the fine folks at dbatools.io for some nifty PowerShell to help us install patches and Brent Ozar for keeping up sqlserverupdates.com. If you have some additional ideas on how to best meet your patching needs, let us know via social media!
The show notes and video for today's episode can be found at https://sqldatapartners.com/2022/08/24/episode-252-the-cumulative-update-dilemma. Have fun on the SQL Trail!
How can you begin to integrate AI models into your environment without spending months building everything out? As we continue our march to Episode 256, we bring back our friend Sam Nasr to talk about Azure Cognitive Services. Sam walks us through a few examples of how he was able to leverage some pre-packaged voice recognition services to assist customers get to the proper area for service. We discuss scenarios where Azure Cognitive Services really shines and where you may have to develop more custom solutions or look for extensions. If you are using Azure Cognitive Services, let us know your experience!
The show notes and video for this episode can be found at https://sqldatapartners.com/2022/08/10/episode-251-azure-cognitive-services. Have fun on the SQL Trail!
This episode we are doing a bit of a look back and are going to air some previously buried content. Some of the bloopers are pre-video and some of them have some physical comedy, which you’ll have to go to YouTube to see. While a few were cut out of the published episodes, they are mostly things you’ve never heard.
The show notes and video for this episode can be found at https://sqldatapartners.com/2022/07/27/episode-250-what-happens-before-companeros/. Have fun on the SQL Trail!