Master SQL Server’s Concurrency Model so you can implement high-throughput systems that deliver transactional consistency to your application customers. This book explains how to troubleshoot and address blocking problems and deadlocks, and write code and design database schemas to minimize concurrency issues in the systems you develop. SQL Server’s Concurrency Model is one of the least understood parts of the SQL Server Database Engine. Almost every SQL Server system experiences hard-to-explain concurrency and blocking issues, and it can be extremely confusing to solve those issues without a base of knowledge in the internals of the Engine. While confusing from the outside, the SQL Server Concurrency Model is based on several well-defined principles that are covered in this book. Understanding the internals surrounding SQL Server’s Concurrency Model helps you build high-throughput systems in multi-user environments. This book guides you through the Concurrency Model and elaborates how SQL Server supports transactional consistency in the databases. The book covers all versions of SQL Server, including Microsoft Azure SQL Database, and it includes coverage of new technologies such as In-Memory OLTP and Columnstore Indexes. What You'll Learn Know how transaction isolation levels affect locking behavior and concurrency Troubleshoot and address blocking issues and deadlocks Provide required data consistency while minimizing concurrency issues Design efficient transaction strategies that lead to scalable code Reduce concurrency problems through good schema design Understand concurrency models for In-Memory OLTP and Columnstore Indexes Reduce blocking during index maintenance, batch data load, and similar tasks Who This Book Is For SQL Server developers, database administrators, and application architects who are developing highly-concurrent applications. The book is for anyone interested in the technical aspects of creating and troubleshooting high-throughput systems that respond swiftly to user requests.

Power BI Data Analysis and Visualization provides a roadmap to vendor choices and highlights why Microsoft’s Power BI is a very viable, cost effective option for data visualization. The book covers the fundamentals and most commonly used features of Power BI, but also includes an in-depth discussion of advanced Power BI features such as natural language queries; embedding Power BI dashboards; and live streaming data. It discusses real solutions to extract data from the ERP application, Microsoft Dynamics CRM, and also offers ways to host the Power BI Dashboard as an Azure application, extracting data from popular data sources like Microsoft SQL Server and open-source PostgreSQL. Authored by Microsoft experts, this book uses real-world coding samples and screenshots to spotlight how to create reports, embed them in a webpage, view them across multiple platforms, and more. Business owners, IT professionals, data scientists, and analysts will benefit from this thorough presentation of Power BI and its functions.

In this podcast Jason Carmel(@defenestrate99) Chief Data Officer @ POSSIBLE talks about his journey leading data analytics practice of digital marketing agency. He sheds light on some methodologies for building a sound data science practice. He sheds light on using data science chops for doing some good while creating traditional value. He shared his perspective on keeping team-high on creativity to keep creating innovative solutions. This is a great podcast for anyone looking to understanding the digital marketing landscape and how to create a sound data science practice.

Timelines: 0:29 Jason's journey. 6:40 Advantage of having a legal background for a data scientist. 9:15 Understanding emotions based on data. 13:54 The empathy model. 14:53 From idea to inception to execution. 23:40 The role of digital agencies. 30:20 Measuring the right amount of data. 32:40 Management in a creative agency. 34:40 Leadership qualities that promote creativity. 38:14 Leader's playbook in a digital agency. 40:50 Qualities of a great data science team in the digital agency. 44:30 Leadership's role in data creativity. 47:00 Opportunites as a data scientist in the digital agency. 49:18 Future of data in digital media. 51:38 Jason's success mantra. 53:30 Jason's favorite reads. 57:11 Key takeaways.

Jason's Recommended Read: Trendology: Building an Advantage through Data-Driven Real-Time Marketing by Chris Kerns amzn.to/2zMhYkV Venomous: How Earth's Deadliest Creatures Mastered Biochemistry by Christie Wilcox amzn.to/2LhqI76

Podcast Link: https://futureofdata.org/jason-carmel-defenestrate99-possible-leading-analytics-data-digital-marketing/

Jason's BIO: Jason Carmel is Chief Data Officer at Possible. With nearly 20 years of digital data and marketing experience, Jason has worked with clients such as Coca Cola, Ford, and Microsoft to evolve digital experiences based on real-time feedback and behavioral data. Jason manages a global team of 100 digital analysts across POSSIBLE, a digital advertising agency that uses traditional and unconventional data sets and models to help brands connect more effectively with their customers.

Of particular interest is Jason’s work using data and machine learning to define and understand the emotional components of human conversation. Jason spearheaded the creation of POSSIBLE’s Empathy Model, with translates the raw, unstructured content of social media into a quantitative understanding of what customers are actually feeling about a given topic, event, or brand.

About #Podcast:

FutureOfData podcast is a conversation starter to bring leaders, influencers and lead practitioners to come on show and discuss their journey in creating the data driven future.

Wanna Join? If you or any you know wants to join in, Register your interest by mailing us @ [email protected]

Want to sponsor? Email us @ [email protected]

Keywords: FutureOfData,

DataAnalytics,

Leadership,

Futurist,

Podcast,

BigData,

Strategy

Identify and fix causes of poor performance. You will learn Query Store, adaptive execution plans, and automated tuning on the Microsoft Azure SQL Database platform. Anyone responsible for writing or creating T-SQL queries will find valuable the insight into bottlenecks, including how to recognize them and eliminate them. This book covers the latest in performance optimization features and techniques and is current with SQL Server 2017. If your queries are not running fast enough and you’re tired of phone calls from frustrated users, then this book is the answer to your performance problems. SQL Server 2017 Query Performance Tuning is about more than quick tips and fixes. You’ll learn to be proactive in establishing performance baselines using tools such as Performance Monitor and Extended Events. You’ll recognize bottlenecks and defuse them before the phone rings. You’ll learn some quick solutions too, but emphasis is on designing for performance and getting it right. The goal is to head off trouble before it occurs. What You'll Learn Use Query Store to understand and easily change query performance Recognize and eliminate bottlenecks leading to slow performance Deploy quick fixes when needed, following up with long-term solutions Implement best practices in T-SQL to minimize performance risk Design in the performance that you need through careful query and index design Utilize the latest performance optimization features in SQL Server 2017 Protect query performance during upgrades to the newer versions of SQL Server Who This Book Is For Developers and database administrators with responsibility for application performance in SQL Server environments. Anyone responsible for writing or creating T-SQL queries will find valuable the insight into bottlenecks, including how to recognize them and eliminate them.

Uncover the power of Microsoft Power BI with this accessible and practical guide. This book introduces you to the concepts of data modeling, transformation, and visualization, ensuring that you can build effective dashboards and gain valuable insights. You'll be empowered to productively utilize Power BI in your organization to achieve your analytics goals. What this Book will help me do Connect to various data sources and harness the capabilities of the Query Editor. Transform and clean data for analysis, learning to use languages like M and R. Build robust data models with relationships and powerful DAX expressions. Create impactful reports with efficient and custom visualizations in Power BI. Deploy and administer Power BI solutions both in the cloud and on-premise. Author(s) The authors, Devin Knight, Mitchell Pearson, and Manuel Quintana, are seasoned experts in Business Intelligence and Power BI. They bring years of experience simplifying complex data challenges. Their writing is approachable and hands-on, equipping readers with the skills to solve real-world problems. Who is it for? This book is perfectly suited for professionals in Business Intelligence roles, data analysts, or those aiming to adopt Power BI solutions. Whether you're new to Power BI or have basic BI knowledge, this guide will take you from fundamentals to advanced implementations. Ideal for anyone aiming to unlock actionable insights from their data.

We revisit the 2018 Microsoft Build in this episode, focusing on the latest ideas in DevOps. Kyle interviews Cloud Developer Advocates Damien Brady, Paige Bailey, and Donovan Brown to talk about DevOps and data science and databases. For a data scientist, what does it even mean to "build"? Packaging and deployment are things that a data scientist doesn't normally have to consider in their day-to-day work. The process of making an AI app is usually divided into two streams of work: data scientists building machine learning models and app developers building the application for end users to consume. DevOps includes all the parties involved in getting the application deployed and maintained and thinking about all the phases that follow and precede their part of the end solution. So what does DevOps mean for data science? Why should you adopt DevOps best practices? In the first half, Paige and Damian share their views on what DevOps for data science would look like and how it can be introduced to provide continuous integration, delivery, and deployment of data science models. In the second half, Donovan and Damian talk about the DevOps life cycle of putting a database under version control and carrying out deployments through a release pipeline.

In this episode, Wayne Eckerson and Jen Underwood explore a new era of analytics. Data volumes and complexity have exceeded the limits of current manual drag-and-drop analytics solutions. Data moves at the speed of light while speed-to-insight lags farther and farther behind. It is time to explore intelligent, next generation, machine-powered analytics to retain your competitive edge. It is time to combine the best of the human mind and machine.

Underwood is an analytics expert and founder of Impact Analytic. She is a former product manager at Microsoft who spearheaded the design and development of the reinvigorated version of Power BI, which has since become a market leading BI tool. Underwood is an IBM Analytics Insider, SAS contributor, former Tableau Zen Master, Top 10 Women Influencer and active analytics community member. She is keenly interested in the intersection of data visualization and data science and writes and speaks persuasively about these topics.

Prepare for Microsoft Exam 70-778-and help demonstrate your real-world mastery of Power BI data analysis and visualization. Designed for experienced BI professionals and data analysts ready to advance their status, Exam Ref focuses on the critical thinking and decision-making acumen needed for success at the MCSA level. Focus on the expertise measured by these objectives: Consume and transform data by using Power BI Desktop Model and visualize data Configure dashboards, reports, and apps in the Power BI Service This Microsoft Exam Ref: Organizes its coverage by exam objectives Features strategic, what-if scenarios to challenge you Assumes you have experience consuming and transforming data, modeling and visualizing data, and configuring dashboards using Excel and Power BI

Why do businesses continue to use Microsoft’s BizTalk Server as the backbone to integrate line-of-business applications with their trading partners and how do recent changes make it even more effective? With the advent of Azure, we have a unique opportunity to enhance BizTalk functionality including reducing the cost of operations and maintenance. This book offers three solutions for the reader on ways to leverage BizTalk to get more from existing deployments or find ways to modernize the deployment via Azure. Microsoft partners are playing a significant role in enhancing the capabilities of BizTalk and this book includes sections that provide an in-depth review of BizTalk 360 © and the WPC HIPAA DB Toolkit ©. Over the recent past, Web 3.0 has also introduced many new concepts and open source technologies and this book covers ways to leverage these to enhance your BizTalk deployment. The authors start with a survey of the existing BizTalk Server – its history, patterns, and state of affairs –and go on to provide an in-depth elaboration of three messaging patterns that customers use for BizTalk; the advantages of updating to SQL Server 2016; a review of partner solutions that enhance BizTalk; and BizTalk with Web 3.0 for custom solutions. The book concludes with a comparison of the three viable BizTalk Azure application solutions that will enable you to make the best choice for your business.

Essential Microsoft® SQL Server® 2017 installation, configuration, and management techniques for Linux Foreword by Kalen Delaney, Microsoft SQL Server MVP This comprehensive guide shows, step-by-step, how to set up, configure, and administer SQL Server 2017 on Linux for high performance and high availability. Written by a SQL Server expert and respected author, Microsoft SQL Server 2017 on Linux teaches valuable Linux skills to Windows-based SQL Server professionals. You will get clear coverage of both Linux and SQL Server and complete explanations of the latest features, tools, and techniques. The book offers clear instruction on adaptive query processing, automatic tuning, disaster recovery, security, and much more. •Understand how SQL Server 2017 on Linux works •Install and configure SQL Server on Linux •Run SQL Server on Docker containers •Learn Linux Administration •Troubleshoot and tune query performance in SQL Server •Learn what is new in SQL Server 2017 •Work with adaptive query processing and automatic tuning techniques •Implement high availability and disaster recovery for SQL Server on Linux •Learn the security features available in SQL Server

Prepare for Microsoft Exam 70-779 and help demonstrate your real-world mastery of Microsoft Excel data analysis and visualization. Designed for BI professionals, data analysts, and others who analyze business data with Excel, this Exam Ref focuses on the critical thinking and decision-making acumen needed for success at the MCSA level. Focus on the expertise measured by these objectives: Consume and transform data by using Microsoft Excel Model data, from building and optimizing data models through creating performance KPIs, actual and target calculations, and hierarchies Visualize data, including creating and managing PivotTables and PivotCharts, and interacting with PowerBI This Microsoft Exam Ref: Organizes its coverage by exam objectives Features strategic, what-if scenarios to challenge you Assumes you have a strong understanding of how to use Microsoft Excel to perform data analysis

Use Microsoft Flow in your business to improve productivity through automation with this step-by-step introductory text from a Microsoft Flow expert. You’ll see the prerequisites to get started with this cloud-based service, including how to create a flow and how to use different connectors. Introducing Microsoft Flow takes you through connecting with SharePoint, creating approval flows, and using mobile apps. This vital information gives you a head-start when planning your Microsoft Flow implementation. The second half of the book continues with managing connections and gateways, where you’ll cover the configuration, creation, and deletion of connectors and how to connect to a data gateway. The final topic is Flow administration and techniques to manage the environment. After reading this book, you will be able to create and manage Flow from desktop, laptop, or mobile devices and connect with multiple services such as SharePoint, Twitter, Facebook, and other networking sites. What You Will Learn Create flows from built-in and blank templates Manage flows, connections, and gateways Create approvals, connect with multiple services, and use mobile apps Who This Book Is For Administrators and those who are interested in creating automated workflows using templates and connecting with multiple services without writing a single line of code.

Summary

Building an ETL pipeline is a common need across businesses and industries. It’s easy to get one started but difficult to manage as new requirements are added and greater scalability becomes necessary. Rather than duplicating the efforts of other engineers it might be best to use a hosted service to handle the plumbing so that you can focus on the parts that actually matter for your business. In this episode CTO and co-founder of Alooma, Yair Weinberger, explains how the platform addresses the common needs of data collection, manipulation, and storage while allowing for flexible processing. He describes the motivation for starting the company, how their infrastructure is architected, and the challenges of supporting multi-tenancy and a wide variety of integrations.

Preamble

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to run a bullet-proof data platform. Go to dataengineeringpodcast.com/linode to get a $20 credit and launch a new server in under a minute. For complete visibility into the health of your pipeline, including deployment tracking, and powerful alerting driven by machine-learning, DataDog has got you covered. With their monitoring, metrics, and log collection agent, including extensive integrations and distributed tracing, you’ll have everything you need to find and fix performance bottlenecks in no time. Go to dataengineeringpodcast.com/datadog today to start your free 14 day trial and get a sweet new T-Shirt. Go to dataengineeringpodcast.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch. Your host is Tobias Macey and today I’m interviewing Yair Weinberger about Alooma, a company providing data pipelines as a service

Interview

Introduction How did you get involved in the area of data management? What is Alooma and what is the origin story? How is the Alooma platform architected?

I want to go into stream VS batch here What are the most challenging components to scale?

How do you manage the underlying infrastructure to support your SLA of 5 nines? What are some of the complexities introduced by processing data from multiple customers with various compliance requirements?

How do you sandbox user’s processing code to avoid security exploits?

What are some of the potential pitfalls for automatic schema management in the target database? Given the large number of integrations, how do you maintain the

What are some challenges when creating integrations, isn’t it simply conforming with an external API?

For someone getting started with Alooma what does the workflow look like? What are some of the most challenging aspects of building and maintaining Alooma? What are your plans for the future of Alooma?

Contact Info

LinkedIn @yairwein on Twitter

Parting Question

From your perspective, what is the biggest gap in the tooling or technology for data management today?

Links

Alooma Convert Media Data Integration ESB (Enterprise Service Bus) Tibco Mulesoft ETL (Extract, Transform, Load) Informatica Microsoft SSIS OLAP Cube S3 Azure Cloud Storage Snowflake DB Redshift BigQuery Salesforce Hubspot Zendesk Spark The Log: What every software engineer should know about real-time data’s unifying abstraction by Jay Kreps RDBMS (Relational Database Management System) SaaS (Software as a Service) Change Data Capture Kafka Storm Google Cloud PubSub Amazon Kinesis Alooma Code Engine Zookeeper Idempotence Kafka Streams Kubernetes SOC2 Jython Docker Python Javascript Ruby Scala PII (Personally Identifiable Information) GDPR (General Data Protection Regulation) Amazon EMR (Elastic Map Reduce) Sequoia Capital Lightspeed Investors Redis Aerospike Cassandra MongoDB

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Support Data Engineering Podcast

There's so much to discuss on the AI side, it's hard to know where to begin. Luckily,  Steve Guggenheimer, Microsoft's corporate vice president of AI Business, and Carlos Pessoa, a software engineering manager for the company's Cloud AI Platform, talked to Kyle about announcements related to AI in industry.

"Practical SQL is an approachable and fast-paced guide to SQL (Structured Query Language), the standard programming language for defining, organizing, and exploring data in relational databases. The book focuses on using SQL to find the story your data tells, with the popular open-source database PostgreSQL and the pgAdmin interface as its primary tools. You’ll first cover the fundamentals of databases and the SQL language, then build skills by analyzing data from the U.S. Census and other federal and state government agencies. With exercises and real-world examples in each chapter, this book will teach even those who have never programmed before all the tools necessary to build powerful databases and access information quickly and efficiently. You’ll learn how to: • Create databases and related tables using your own data• Define the right data types for your information• Aggregate, sort, and filter data to find patterns• Use basic math and advanced statistical functions• Identify errors in data and clean them up• Import and export data using delimited text files• Write queries for geographic information systems (GIS)• Create advanced queries and automate tasks Learning SQL doesn’t have to be dry and complicated. Practical SQL delivers clear examples with an easy-to-follow approach to teach you the tools you need to build and manage your own databases. This book uses PostgreSQL, but the SQL syntax is applicable to many database applications, including Microsoft SQL Server and MySQL."

Create compelling business infographics with SAS and familiar office productivity tools. A picture is worth a thousand words, but what if there are a billion words? When analyzing big data, you need a picture that cuts through the noise. This is where infographics come in. Infographics are a representation of information in a graphic format designed to make the data easily understandable. With infographics, you don’t need deep knowledge of the data. The infographic combines story telling with data and provides the user with an approachable entry point into business data. Infographics Powered by SAS : Data Visualization Techniques for Business Reporting shows you how to create graphics to communicate information and insight from big data in the boardroom and on social media. Learn how to create business infographics for all occasions with SAS and learn how to build a workflow that lets you get the most from your SAS system without having to code anything, unless you want to! This book combines the perfect blend of creative freedom and data governance that comes from leveraging the power of SAS and the familiarity of Microsoft Office. Topics covered in this book include: SAS Visual Analytics SAS Office Analytics SAS/GRAPH software (SAS code examples) Data visualization with SAS Creating reports with SAS Using reports and graphs from SAS to create business presentations Using SAS within Microsoft Office

In this podcast, Wayne Eckerson and Joe Caserta discuss data migration, compare cloud offerings from Amazon, Google, and Microsoft, and define and explain artificial intelligence.

You can contact Caserta by visiting caserta.com or by sending him an email to [email protected]. Follow him on Twitter @joe_caserta.

Caserta is President of a New York City-based consulting firm he founded in 2001 and a longtime data guy. In 2004, Joe teamed up with data warehousing legend, Ralph Kimball to write to write the book The Data Warehouse ETL Toolkit. Today he’s now one of the leading authorities on big data implementations. This makes Joe one of the few individuals with in-the-trenches experience on both sides of the data divide, traditional data warehousing on relational databases and big data implementations on Hadoop and the cloud.

In this podcast, Wayne Eckerson and James Serra discuss myths of modern data management. Some of the myths discussed include 'all you need is a data lake', 'the data warehouse is dead', 'we don’t need OLAP cubes anymore', 'cloud is too expensive and latency is too slow', 'you should always use a NoSQL product over a RDBMS.'

Serra is big data and data warehousing solutions architect at Microsoft with over thirty years of IT experience. He is a popular blogger and speaker and has presented at dozens of Microsoft PASS and other events. Prior to Microsoft, Serra was an independent data warehousing and business intelligence architect and developer.

Dive right into the powerful world of Microsoft Power BI with this comprehensive guide. This book takes you through every step of mastering Power BI, from data modeling to creating actionable visualizations. You'll find clear explanations and practical steps to improve your data analytics and enhance business decision-making. What this Book will help me do Learn to connect and transform data using Power Query M Language to create clean, structured datasets. Understand how to design scalable and performance-optimized Power BI Data Models for effective analytics. Develop professional, visually appealing and interactive reports and dashboards to convey insights confidently. Implement best practices for managing Power BI solutions, including deployment, version control, and monitoring. Gain practical knowledge to administer Power BI across organizational structures, ensuring security and efficiency. Author(s) None Powell is a seasoned expert in business intelligence and a passionate educator in the field of data analytics. With extensive hands-on experience in Microsoft Power BI, None has supported many organizations in unlocking the potential of their data. The approachable writing style reflects a real-world yet proficient understanding of Power BI's capabilities. Who is it for? This book is ideal for business intelligence professionals looking to deepen their expertise in Microsoft Power BI. Readers already familiar with basic BI concepts and Power BI will gain significant technical depth. It suits professionals keen to enhance their data modeling, visualization, and analytics skills. If you're aiming to create impactful dashboards and benefit from advanced insights, this book is for you.

Some people (possibly even one of the co-hosts of this podcast...on this very episode) have been known to say, "People have this dependency on Excel, which is freakin' weird!" We know it wasn't Tim, because he wouldn't have filtered his language! Whether it's a symptom of weirdness, an illustration of inertia, or an invisible hand of inevitability, though, Excel remains omnipresent. Is that a good thing? Is it a bad thing? Is it merely "a thing?" In this episode, the gang dives into the topic: the good and bad of Excel, the various paths to a future where its ubiquity is no longer a given, and different strategies and considerations for moving towards that future.  For complete show notes, including links to items mentioned in this episode and a transcript of the show, visit the show page.