Stay Online When the Grid Dies! When the power or internet fails, your workflow shouldn’t. In 13 minutes, I’ll demo a Ruby-powered importer that syncs GitHub activity to Mastodon, turning contribution graphs into animated Conway’s Game of Life SVGs published to the Fediverse. 100 % offline-capable. Join to brainstorm a blackout-resilient dev stack and maybe co-build the next piece.

CoSApp, for Collaborative System Approach, is a Python library dedicated to the simulation and design of multi-disciplinary systems. It is primarily intended for engineers and system architects during the early stage of industrial product design. The API of CoSApp is focused on simplicity and explicit declaration of design problems. Special attention is given to modularity; a very flexible mechanism of solver assembly allows users to construct complex, customized simulation workflows. This presentation aims at presenting the key features of the framework.

https://cosapp.readthedocs.io https://gitlab.com/cosapp/cosapp

As your organization scales to 20+ data science teams and 300+ DS/ML/DE engineers, you face a critical challenge: how to build a secure, reliable, and scalable orchestration layer that supports both fast experimentation and stable production workflows. We chose Airflow — and didn’t regret it! But to make it truly work at our scale, we had to rethink its architecture from the ground up. In this talk, we’ll share how we turned Airflow into a powerful MLOps platform through its core capability: running pipelines across multiple K8s GPU clusters from a single UI (!) using per-cluster worker pools. To support ease of use, we developed MLTool — our own library for fast and standardized DAG development, integrated Vault for secure secret management across teams, enabled real-time logging with S3 persistence and built a custom SparkSubmitOperator for Kerberos-authenticated Spark/Hadoop jobs in Kubernetes. We also streamlined the developer experience — users can generate a GitLab repo and deploy a versioned pipeline to prod in under 10 minutes! We’re proud of what we’ve built — and our users are too. Now we want to share it with the world!

At Zillow, we have accelerated the volume and quality of our dashboards by leveraging a modern SDLC with version control and CI/CD. In the past three months, we have released 32 production-grade dashboards and shared them securely across the organization while cutting error rates in half over that span. In this session, we will provide an overview of how we utilize Databricks asset bundles and GitLab CI/CD to create performant dashboards that can be confidently used for mission-critical operations. As a concrete example, we'll then explore how Zillow's Data Platform team used this approach to automate our on-call support analysis, leveraging our dashboard development strategy alongside Databricks LLM offerings to create a comprehensive view that provides actionable performance metrics alongside AI-generated insights and action items from the hundreds of requests that make up our support workload.

This session shows how engineers can use Gemini Cloud Assist and Gemini Code Assist to speed up the software development life cycle (SDLC) and improve service quality. You’ll learn how to shorten release cycles; improve delivery quality with best practices and generated code, including tests and infrastructure as code (IaC); and gain end-to-end visibility into service setup, consumption, cost, and observability. In a live demo, we’ll showcase the integrated flow and highlight code generation with GitLab and Jira integration. And we’ll show how Gemini Cloud Assist provides deeper service-quality insights.

Code assist tools are transforming software development, enhancing productivity with intelligent suggestions and automation. Yet they also pose challenges in ensuring code security, managing observability, and addressing risks from automation. Join experts from Google Cloud, Datadog, GitLab, Harness, and Snyk in a dynamic panel as they explore the potential of code assist tools and share strategies to mitigate risks, safeguard workflows, and maximize the impact of these tools in today’s fast-paced development landscape.

Discover how Renault transformed automotive software development (SDV) with Google Cloud. By replacing physical prototypes with Android-based virtualization, they accelerated their SDV life cycle and moved to a cloud-first, iterative approach. Learn how they leverage Cloud Workstations, Gemini Code Assist, and a continuous integration and continuous testing (CI/CT) pipeline powered by Google Kubernetes Engine and GitLab to boost developer productivity and bring new features to market faster.

Jupyter based environments are getting a lot of traction for teaching computing, programming, and data sciences. The narrative structure of notebooks has indeed proven its value for guiding each student at it's own pace to the discovery and understanding of new concepts or new idioms (e.g. how do I extract a column in pandas?). But then these new pieces of knowledge tend to quickly fade out and be forgotten. Indeed long term acquisition of knowledge and skills takes reinforcement by repetition. This is the foundation of many online learning platforms like Webwork or WIMS that offer exercises with randomization and automatic feedback. And of popular "AI-powered" apps -- e.g. to learn foreign languages -- that use spaced repetition algorithms designed by educational and neuro sciences to deliver just the right amount of repetition.

What if you could author such exercizes as notebooks, to benefit from everything that Jupyter can offer (think rich narratives, computations, visualization, interactions)? What if you could integrate such exercises right into your Jupyter based course? What if a learner could get personalized exercise recommandations based on their past learning records, without having to give away these sensitive pieces of information away?

That's Jupylates (work in progress). And thanks to the open source scientific stack, it's just a small Jupyter extension.

Using various operators to perform daily routines. Integration with Technologies: Redis: Acts as a caching mechanism to optimize data retrieval and processing speed, enhancing overall pipeline performance. MySQL: Utilized for storing metadata and managing task state information within Airflow’s backend database. Tableau: Integrates with Airflow to generate interactive visualizations and dashboards, providing valuable insights into the processed data. Amazon Redshift: Panasonic leverages Redshift for scalable data warehousing, seamlessly integrating it with Airflow for data loading and analytics. Foundry: Integrated with Airflow to access and process data stored within Foundry’s data platform, ensuring data consistency and reliability. Plotly Dashboards: Employed for creating custom, interactive web-based dashboards to visualize and analyze data processed through Airflow pipelines. GitLab CI/CD Pipelines: Utilized for version control and continuous integration/continuous deployment (CI/CD) of Airflow DAGs (Directed Acyclic Graphs), ensuring efficient development and deployment of workflows.

Join this session to discover how you can deliver software, faster with GitLab Duo. You will learn more about: - How the power of AI will drive enhancements across your entire software development lifecycle and - The next generation of GitLab Duo By attending this session, your contact information may be shared with the sponsor for relevant follow up for this event only.

Click the blue “Learn more” button above to tap into special offers designed to help you implement what you are learning at Google Cloud Next 25.

Dive into GitLab Duo's integration with Google Cloud to enhance your DevSecOps strategy. We'll demonstrate how GitLab Duo leverages Vertex AI for smarter, faster app development and deployment. This session offers direct insights into improving app security and efficiency in your cloud projects, showcasing GitLab Duo’s unique capabilities within the Google Cloud ecosystem. Perfect for developers and IT professionals seeking practical, scalable solutions for their software delivery challenges.

Click the blue “Learn more” button above to tap into special offers designed to help you implement what you are learning at Google Cloud Next 25.

Join this session to discover how you can deliver software, faster with GitLab Duo. You will learn more about: - How the power of AI will drive enhancements across your entire software development lifecycle and - The next generation of GitLab Duo By attending this session, your contact information may be shared with the sponsor for relevant follow up for this event only.

Click the blue “Learn more” button above to tap into special offers designed to help you implement what you are learning at Google Cloud Next 25.

GitLab and Google Cloud are partnering to deliver a comprehensive, integrated DevSecOps solution that provides best-in-class reliability, efficiency and end-to-end security. In this session you'll learn how you can quickly and securely deploy workloads to Google Cloud using GitLab’s new integrations, which include streamlined IAM configuration, Artifact Registry integration, and optimized GitLab templates and workflows for Google Cloud.

Click the blue “Learn more” button above to tap into special offers designed to help you implement what you are learning at Google Cloud Next 25.

ABOUT THE TALK: When Taylor Murphy joined GitLab, they had just raised their Series C, had about 200 people, and he was the only person "doing data." Over the next 3 years, the company would 6x its total headcount and be on target to IPO, which it did in 2021, all while the demand for data and insights grew exponentially. This talk will detail that growth journey with a particular focus on how they built the data culture across the organization. Taylor will share what went well and what he would repeat, and he'll be honest about what he would do differently if he could go back in time and do it all again.

ABOUT THE SPEAKER: Taylor Murphy is the Head of Product and Data at Meltano, an open source data platform that enables collaboration, efficiency, and visibility. Taylor has been deeply involved in leading and building data-informed teams his entire career.

At Concert Genetics he scaled the Data Operations team to enable the management of hundreds of thousands of genetic tests and millions of claims records.

At GitLab, he was the first data hire where he focused on building and scaling the data organization as the company headed towards its IPO.

ABOUT DATA COUNCIL: Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers.

Make sure to subscribe to our channel for the most up-to-date talks from technical professionals on data related topics including data infrastructure, data engineering, ML systems, analytics and AI from top startups and tech companies.

FOLLOW DATA COUNCIL: Twitter: https://twitter.com/DataCouncilAI LinkedIn: https://www.linkedin.com/company/datacouncil-ai/

How do you support exponentially growing companies without breaking as a data team? The answer is increasing your leverage with tools and processes. This session centers around four principles to achieve this goal: 1. don’t reinvent the wheel, 2. make your own job easier, 3. save time for innovation, and 4. invest in onboarding.

First, the first data leader at Vendr, the SaaS buying platform with customers like GitLab, Brex, and The Washington Post, will share his learnings on building a stack and team that scaled as the company grew 10x from 30 to 300 employees in under two years.

Second, we’ll give a demo of how Metaplane pulls lineage and metadata from a modern data stack that is centered around dbt. By the end of the demo, you’ll know how to setup tests, extract lineage throughout your data stack, and triage data quality alerts.More details coming soon!

Check the slides here: https://docs.google.com/presentation/d/15dQJIGeGhG0WGO6MLXtxWhmf8neY-u0c8ZLRG9GJB-s/edit?usp=sharing

Coalesce 2023 is coming! Register for free at https://coalesce.getdbt.com/.

Discover how to manage and optimize the life cycle of your data science projects with Comet! By the end of this book, you will master preparing, analyzing, building, and deploying models, as well as integrating Comet into your workflow. What this Book will help me do Master managing data science workflows with Comet. Confidently prepare and analyze your data for effective modeling. Deploy and monitor machine learning models using Copet tools. Integrate Comet with DevOps and GitLab workflows for production readiness. Apply Comet to advanced topics like NLP, deep learning, and time series analysis. Author(s) Angelica Lo Duca is an experienced author and data scientist with years of expertise in data science workflows and tools. She brings practical insights into integrating platforms like Comet into modern data science tasks. Who is it for? If you are a data science practitioner or programmer looking to understand and implement efficient project lifecycles using Comet, this book is tailored for you. A basic backdrop in data science and programming is highly recommended, but prior expertise in Comet is unnecessary.

Send us a text Want to be featured as a guest on Making Data Simple? Reach out to us at [[email protected]] and tell us why you should be next.

Abstract Making Data Simple Podcast is hosted by Al Martin, VP, IBM Expert Services Delivery, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun. This week on Making Data Simple, we have Douwe Maan. Douwe is the Founder and CEO of Meltano, an open source data integration and transformation platform. He joined GitLab as employee number 10 while still in college and later became its Engineering Manager. Meltano originally began as an internal project within GitLab, but spun out as an independent startup in early 2021 while raising $4.2 million in seed funding led by GV. Today, Meltano has over 5,000 active projects every month and supports data integration connectors for almost 300 sources and destinations. Show Notes 1:30 – Douwe’s history 4:00 – What is an iPhone jail break? 8:04 – How would you describe Meltano? 10:38 – What kind of tools do you use? 12:12 – Did you start with ELT? 16:08 – Is this dev ops or data ops and what is the difference? 18:38 – What platform does Meltano use? 29:52 – What can you do that no one else can do? 32:10 – Do you have point and click analytics? 36:23 – How do you measure success? Meltano  Connect with the Team Producer Kate Brown - LinkedIn. Producer Steve Templeton - LinkedIn. Host Al Martin - LinkedIn and Twitter.  Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

Join this session to discover how you can deliver software, faster with GitLab Duo. You will learn more about: - How the power of AI will drive enhancements across your entire software development lifecycle and - The next generation of GitLab Duo By attending this session, your contact information may be shared with the sponsor for relevant follow up for this event only.

Click the blue “Learn more” button above to tap into special offers designed to help you implement what you are learning at Google Cloud Next 25.