In this episode, Conor and Bryce chat about some open source projects, podcast recommendations, our upcoming trip to Europe and much more! Link to Episode 249 on WebsiteDiscuss this episode, leave a comment, or ask a question (on GitHub)Socials ADSP: The Podcast: TwitterConor Hoekstra: Twitter | BlueSky | MastodonBryce Adelstein Lelbach: TwitterShow Notes Date Recorded: 2025-08-21 Date Released: 2025-08-29 Astro Bot VideoADSP Episode 176: 🇺🇸 prior, deltas & Dinner with PhineasThrust Github Search Vibing ProjectPaddlePaddle/Paddle RepoUber AresDB RepoLatent Space PodcastBig Technology PodcastCheeky Pint PodcastDwarkesh PodcastTraining Data PodcastADSP Episode 39: How Steve Jobs Saved Sean ParentRoku Engineering SymposiumCopenhagen C++ MeetupCasey Muratori – The Big OOPs: Anatomy of a Thirty-five-year Mistake – BSC 2025NDC Tech Town CUDA Python WorkshopNDC Tech Town CUDA C++ WorkshopIntro Song Info Miss You by Sarah Jansen https://soundcloud.com/sarahjansenmusic Creative Commons — Attribution 3.0 Unported — CC BY 3.0 Free Download / Stream: http://bit.ly/l-miss-you Music promoted by Audio Library https://youtu.be/iYYxnasvfx8

In this episode, Conor and Ben chat about the philosophy of good software design, TV shows, movies and more! Link to Episode 248 on WebsiteDiscuss this episode, leave a comment, or ask a question (on GitHub)Socials ADSP: The Podcast: TwitterConor Hoekstra: Twitter | BlueSky | MastodonBen Deane: Twitter | BlueSkyShow Notes Date Recorded: 2025-08-05 Date Released: 2025-08-22 MISRA C++Lightning Talk: Strategies for Developing Safety-Critical Software in C++ - Emily Durie-JohnsonSkeuomorphismC++Now 2018: Ben Deane “Easy to Use, Hard to Misuse: Declarative Style in C++”Intro Song Info Miss You by Sarah Jansen https://soundcloud.com/sarahjansenmusic Creative Commons — Attribution 3.0 Unported — CC BY 3.0 Free Download / Stream: http://bit.ly/l-miss-you Music promoted by Audio Library https://youtu.be/iYYxnasvfx8

In this episode, We talked with Pastor, a medical doctor who built a career in machine learning while studying medicine. Pastor shares how he balanced both fields, leveraged live courses and public sharing to grow his skills, and found opportunities through freelancing and mentoring.TIMECODES00:00 Pastor’s background and early programming journey06:05 Learning new tools and skills on the job while studying medicine11:44 Balancing medical studies with data science work and motivation13:48 Applying medical knowledge to data science and vice versa18:44 Starting freelance work on Upwork and overcoming language challenges24:03 Joining the machine learning engineering course and benefits of live cohorts27:41 Engaging with the course community and sharing progress publicly35:16 Using LinkedIn and social media for career growth and interview opportunities41:03 Building reputation, structuring learning, and leveraging course projects50:53 Volunteering and mentoring with DeepLearning.AI and Stanford Coding Place57:00 Managing time and staying productive while studying medicine and machine learningConnect with Pastor Twitter - https://x.com/PastorSotoB1Linkedin -   / pastorsoto  Github - https://github.com/sotoblancoWebsite - https://substack.com/@pastorsotoConnect with DataTalks.Club: Join the community - https://datatalks.club/slack.htmlSubscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/...Check other upcoming events - https://lu.ma/dtc-eventsGitHub: https://github.com/DataTalksClubLinkedIn -   / datatalks-club   Twitter -   / datatalksclub   Website - https://datatalks.club/

Struggling with data trust issues, dashboard drama, or constant pipeline firefighting? In this deep‑dive interview, Lior Barak shows you how to shift from a reactive “fix‑it” culture to a mindful, impact‑driven practice rooted in Zen/Wabi‑Sabi principles. You’ll learn: Why 97 % of CEOs say they use data, but only 24 % call themselves data‑driven The traffic‑light dashboard pattern (green / yellow / red) that instantly tells execs whether numbers are safe to use A practical rule for balancing maintenance, rollout, and innovation—and avoiding team burnout How to quantify ROI on data products, kill failing legacy systems, and handle ad‑hoc exec requests without derailing roadmaps Turning “imperfect” data into business value with mindful communication, root‑cause logs, and automated incident review loops

🕒 TIMECODES 00:00 Community and mindful data strategy 04:06 Career journey and product management insights 08:03 Wabi-sabi data and the trust crisis 11:47 AI, data imperfection, and trust challenges 20:05 Trust crisis examples and root cause analysis 25:06 Regaining trust through mindful data management 30:47 Traffic light system and effective communication 37:41 Communication gaps and team workload balance 39:58 Maintenance stress and embracing Zen mindset 49:29 Accepting imperfection and measuring impact 56:19 Legacy systems and managing executive requests 01:00:23 Role guidance and closing reflections

🔗 Connect with Lior LinkedIn - https://www.linkedin.com/in/liorbarak Website - https://cookingdata.substack.com/ Cooking Data newsletter: https://cookingdata.substack.com/ Product product lifecycle manager: https://app--data-product-lifecycle-manager-c81b10bb.base44.app/

🔗 Connect with DataTalks.Club Join the community - https://datatalks.club/slack.html Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/u/0/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ Check other upcoming events - https://lu.ma/dtc-events GitHub: https://github.com/DataTalksClub LinkedIn - https://www.linkedin.com/company/datatalks-club/ Twitter - https://x.com/DataTalksClub Website - https://datatalks.club/

🔗 Connect with Alexey Twitter - https://x.com/Al_Grigor Linkedin - https://www.linkedin.com/in/agrigorev/

In this episode, Conor and Ben chat about the philosophy of good software design, learning languages and more! Link to Episode 247 on WebsiteDiscuss this episode, leave a comment, or ask a question (on GitHub)Socials ADSP: The Podcast: TwitterConor Hoekstra: Twitter | BlueSky | MastodonBen Deane: Twitter | BlueSkyShow Notes Date Recorded: 2025-08-05 Date Released: 2025-08-15 非诚勿扰 (Fei Chang Wu Rao) TV ShowSTLab Videos (Adobe Training)Sean Parent TalksA9 VideosSoftware Engineering Languages - Titus Winters - CppNorth 2022C++: Engineers Wanted, Programmers not so Much - David Sankel - C++Now 2019Intro Song Info Miss You by Sarah Jansen https://soundcloud.com/sarahjansenmusic Creative Commons — Attribution 3.0 Unported — CC BY 3.0 Free Download / Stream: http://bit.ly/l-miss-you Music promoted by Audio Library https://youtu.be/iYYxnasvfx8

Microsoft Fabric was announced during Microsoft Build 2023. Which caused a lot of excitement in the Microsoft Data Platform community. Last year, the ability to configure GitHub as a provider for Git integration was introduced. In this session I give a brief overview of Microsoft Fabric before providing a basic introduction to GitHub. I then into details about how you can use GitHub with Microsoft Fabric at this moment in time. Including Git integration and other ways that you can connect up to the Data Warehouse experience. During the session I will also introduce some GitHub best practices. Including naming conventions and security. Plus, show some demos. By the end of this session, you will have a better understanding of how you can use GitHub with Microsoft Fabric.

Building a semantic model via GitHub Copilot (Agent mode)

In this episode, Conor gets Ben's thoughts on AI! Link to Episode 246 on WebsiteDiscuss this episode, leave a comment, or ask a question (on GitHub)Socials ADSP: The Podcast: TwitterConor Hoekstra: Twitter | BlueSky | MastodonBen Deane: Twitter | BlueSkyShow Notes Date Recorded: 2025-08-05 Date Released: 2025-08-08 2025 Stack Overflow Developer SurveyADSP Episode 244: High on AI (Part 1) DiscussionMeasuring the Impact of Early-2025 AI on Experienced Open-Source Developer ProductivitySoftware Unscript Episode 109: GPU Programming and Language Design with Chris LattnerDeclarative Style Evolved - Declarative Structure - Ben Deane - C++Now 2025Intro Song Info Miss You by Sarah Jansen https://soundcloud.com/sarahjansenmusic Creative Commons — Attribution 3.0 Unported — CC BY 3.0 Free Download / Stream: http://bit.ly/l-miss-you Music promoted by Audio Library https://youtu.be/iYYxnasvfx8

In this episode, we talk with Orell about his journey from electrical engineering to freelancing in data engineering. Exploring lessons from startup life, working with messy industrial data, the realities of freelancing, and how to stay up to date with new tools.

Topics covered: Why Orel left a PhD and a simulation‑focused start‑up after Covid hitWhat he learned trying (and failing) to commercialise medical‑imaging simulationsThe first freelance project and the long, quiet months that followedHow he now finds clients, keeps projects small and delivers value quicklyTypical work he does for industrial companies: parsing messy machine logs, building simple pipelines, adding structure laterFavorite everyday tools (Python, DuckDB, a bit of C++) and the habit of blocking time for learningAdvice for anyone thinking about freelancing: cash runway, networking, and focusing on problems rather than “perfect” tech choices A practical conversation for listeners who are curious about moving from research or permanent roles into freelance data engineering.

🕒 TIMECODES 0:00 Orel’s career and move to freelancing 9:04 Startup experience and data engineering lessons 16:05 Academia vs. startups and starting freelancing 25:33 Early freelancing challenges and networking 34:22 Freelance data engineering and messy industrial data 43:27 Staying practical, learning tools, and growth 50:33 Freelancing challenges and client acquisition 58:37 Tools, problem-solving, and manual work

🔗 CONNECT WITH ORELL Twitter - https://bsky.app/profile/orgarten.bsk... LinkedIn - / ogarten
Github - https://github.com/orgarten Website - https://orellgarten.com

🔗 CONNECT WITH DataTalksClub Join the community - https://datatalks.club/slack.html Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/... Check other upcoming events - https://lu.ma/dtc-events GitHub: https://github.com/DataTalksClub LinkedIn - / datatalks-club
Twitter - / datatalksclub
Website - https://datatalks.club/

🔗 CONNECT WITH ALEXEY Connect with Alexey Twitter - / al_grigor
Linkedin - / agrigorev

In this episode, Conor and Bryce continue part 2 of their chat about AI, how it's changing the way they work and more. Link to Episode 245 on WebsiteDiscuss this episode, leave a comment, or ask a question (on GitHub)Socials ADSP: The Podcast: TwitterConor Hoekstra: Twitter | BlueSky | MastodonBryce Adelstein Lelbach: TwitterShow Notes Date Recorded: 2025-07-01 Date Released: 2025-08-01 AI Poll ResultsAll of Conor's Vibe Coded ProjectsCursorClaude 4C++Now 2019: Hana Dusíková “Compile Time Regular Expressions with A Deterministic Finite Automaton”GPU ModeIntro Song Info Miss You by Sarah Jansen https://soundcloud.com/sarahjansenmusic Creative Commons — Attribution 3.0 Unported — CC BY 3.0 Free Download / Stream: http://bit.ly/l-miss-you Music promoted by Audio Library https://youtu.be/iYYxnasvfx8

How do you let any MCP-aware agent tap into your platform? Build a remote server! In 30 minutes, Toby P. provides first-hand insights behind building GitHub’s remote MCP server. Then Joe Z. will follow up by building up a fresh remote server live, using the open-source MCP SDK. Walk away with a clear blueprint to launch your own remote MCP server before the day is done.

Let's learn how to create and update database schemas using just prompts with GitHub Copilot in VS Code. You will see an example of using GibsonAI MCP together with GitHub MCP Servers to create GitHub pull requests with auto-generated model classes in any programming language.

Join Jay Parikh, Microsoft EVP of Core AI, as he opens MCP DevDays with an exciting look at how the Model Context Protocol is revolutionizing AI application development. Discover why Microsoft is all-in on MCP and how it's accelerating developer productivity across VS Code, GitHub Copilot, Azure AI Foundry, and Windows. This keynote features lightning demos showcasing real-world MCP implementations. Whether you're a developer, tool builder, or AI enthusiast, this session sets the stage for two days of hands-on learning about the protocol that's defining the next generation of intelligent

In this episode, Conor and Bryce chat about AI, how it's changing the way we work and more. Link to Episode 244 on WebsiteDiscuss this episode, leave a comment, or ask a question (on GitHub)Socials ADSP: The Podcast: TwitterConor Hoekstra: Twitter | BlueSky | MastodonBryce Adelstein Lelbach: TwitterShow Notes Date Generated: 2025-07-01 Date Released: 2025-07-25 AI Poll ResultsAll of Conor's Vibe Coded ProjectsCursorClaude 4Vittorio's CamomillaGPU ModeADSP Episode 238: Recommended Podcast Discussions on AI & LLMsADSP Episode 239: Claude-Poisoned Dev Sipping Rocket FuelCoRecursive Episode 113: When AI Codes, What’s Left for me?My AI Skeptic Friends Are All Nuts - Thomas PtacekThePrimeTime - How WE Use AI In Software DevelopmentIntro Song Info Miss You by Sarah Jansen https://soundcloud.com/sarahjansenmusic Creative Commons — Attribution 3.0 Unported — CC BY 3.0 Free Download / Stream: http://bit.ly/l-miss-you Music promoted by Audio Library https://youtu.be/iYYxnasvfx8

Kickoff session for Week 1: Setting up your coding environment and GitHub, choosing a project using updated guides, and committing to building for the 7-week Build & Learn program. Hosted by Lindsey, senior data scientist.

In this episode, Conor and Bryce chat about language learning apps, recent C++/CUDA/Python meetups and more! Link to Episode 243 on WebsiteDiscuss this episode, leave a comment, or ask a question (on GitHub)Socials ADSP: The Podcast: TwitterConor Hoekstra: Twitter | BlueSky | MastodonBryce Adelstein Lelbach: TwitterShow Notes Date Generated: 2025-07-01 Date Released: 2025-07-18 MondlyduolingoBabbelADSP Episode 213: NumPy & Summed-Area TablesADSP Episode 227: Re: The CUDA C++ Developer’s ToolboxIntro Song Info Miss You by Sarah Jansen https://soundcloud.com/sarahjansenmusic Creative Commons — Attribution 3.0 Unported — CC BY 3.0 Free Download / Stream: http://bit.ly/l-miss-you Music promoted by Audio Library https://youtu.be/iYYxnasvfx8

Come join the BoF to do a practice run on contributing to a GitHub project. We will walk through how to open a Pull Request for a bugfix, using the workflow most libraries participating at the weekend sprints use (hosted by the sprint chairs)

The rapid growth of scientific data repositories demands innovative solutions for efficient metadata creation. In this talk, we present our open-source project that leverages large language models to automate the generation of standard-compliant metadata files from raw scientific datasets. Our approach harnesses the capabilities of pre-trained open source models, finetuned with domain-specific data, and integrated with Langgraph to orchestrate a modular, end-to-end pipeline capable of ingesting heterogeneous raw data files and outputting metadata conforming to specific standards.

The methodology involves a multi-stage process where raw data is first parsed and analyzed by the LLM to extract relevant scientific and contextual information. This information is then structured into metadata templates that adhere strictly to recognized standards, thereby reducing human error and accelerating the data release cycle. We demonstrate the effectiveness of our approach using the USGS ScienceBase repository, where we have successfully generated metadata for a variety of scientific datasets, including images, time series, and text data.

Beyond its immediate application to the USGS ScienceBase repository, our open-source framework is designed to be extensible, allowing adaptation to other data release processes across various scientific domains. We will discuss the technical challenges encountered, such as managing diverse data formats and ensuring metadata quality, and outline strategies for community-driven enhancements. This work not only streamlines the metadata creation workflow but also sets the stage for broader adoption of generative AI in scientific data management.

Additional Material: - Project supported by USGS and ORNL - Codebase will be available on GitHub after paper publication - Fine-tuned LLM models will be available on Hugginface after paper publication

Would you rather read a “Climate summary” or a “Climate summary for exactly where you live”? Producing documents that tailor your scientific results to an individual or their situation increases understanding, engagement, and connection. But, producing many reports can be onerous.

If you are looking for a way to automate producing many reports, or you produce reports like this but find yourself in copy-and-paste hell, come along to learn how Quarto solves this problem with parameterized reports - you create a single Python notebook, but you generate many beautiful customized PDFs.

Slides

The SciPy Proceedings (https://proceedings.scipy.org) have long served as a cornerstone for publishing research in the scientific python community; with over 330 peer-reviewed articles being published over the last 17 years. In 2024, the SciPy Proceedings underwent a significant transformation, adopting MyST Markdown (https://mystmd.org) and Curvenote (https://curvenote.com) to enhance accessibility, interactivity, and reproducibility — including publishing of Jupyter Notebooks. The new proceedings articles are web-first, providing features such as deep-dive links for cross-references and previews of GItHub content, interactive 3D visualizations, and rich-rendering of Jupyter Notebooks. In this talk, we will (1) present the new authoring & reading capabilities introduced in 2024; (2) highlight connections to prominent open-science initiatives and their impact on advancing computational research publishing; and (3) demonstrate the underlying technologies and how they enhance integrations with SciPy packages and how to use these tools in your own communication workflows.

Our presentation will give an overview of the revised authoring process for SciPy Proceedings; how we improve metadata standards in a similar way to code-linting and continuous integration; and the integration of live previews of the articles, including auto-generated PDFs and JATS XML (a standard used in scientific publishing). The peer-review process for the proceedings currently happens using GitHub’s peer-review commenting in a similar fashion to the Journal of Open Source Software; we will demonstrate this process as well as showcase opportunities for working with distributed review services such as PREreview (https://prereview.org). The open publishing pipeline has streamlined the submission, review, and revision processes while maintaining high scientific quality and improving the completeness of scholarly metadata. Finally, we will present how this work connects into other high-profile scientific publishing initiatives that have incorporated Jupyter Notebooks and live computational figures as well as interactive displays of large-scale data. These initiatives include Notebooks Now! by the American Geophysical Union, which is focusing on ensuring that Jupyter Notebooks can be properly integrated into the scholarly record; and the Microscopy Society of America’s work on interactive publishing and publishing of large-scale microscopy data with interactive visualizations. These initiatives and the SciPy Proceedings are enabled by recent improvements in open-source tools including MyST Markdown, JupyterLab, BinderHub, and Curvenote, which enable new ways to share executable research content. These initiatives collectively aim to improve both the reproducibility, interactivity, and the accessibility of research by providing improved connections between data, software and narrative research articles.

By embracing open science principles and modern technologies, the SciPy Proceedings exemplify how computational research can be more transparent, reproducible, and accessible. The shift to computational publishing, especially in the context of the scientific python community, opens new opportunities for researchers to publish not only their final results but also the computational workflows, datasets, and interactive visualizations that underpin them. This transformation aligns with broader efforts in open science infrastructure, such as integrating persistent identifiers (DOIs, ORCID, ROR), and adopting FAIR (Findable, Accessible, Interoperable, Reusable) principles for computational content. Building on these foundations, as well as open tools like MyST Markdown and Curvenote, provides a scalable model for open scientific publishing that bridges the gap between computational research and scholarly communication, fostering a more collaborative, iterative, and continuous approach to scientific knowledge dissemination.