talk-data.com

Nouvelle édition, même ADN : concret, zéro bullshit. Au menu : 2 talks qui collent au terrain.

1) Évaluation continue d’agents IA en production (Wakam) Déployer des agents IA en production c'est bien, mais comment s'assurer qu'ils ne régressent pas au fil du temps ? Chez Wakam, on a construit une plateforme complète d'évaluation qui combine :

• Génération automatique de datasets synthétiques via des agents IA spécialisés dans Dust
• Validation humaine par les experts métiers dans une interface dédiée via Retool
• Orchestration des évaluations via Prefect
• Monitoring continu dans Langfuse

On vous montrera notre architecture et stack (Dust.tt), Retool, Prefect, Langfuse) et comment elle résout deux pain points majeurs : l'absence d'évaluation native dans les plateformes d'agents SaaS et la complexité de maintenir des datasets à jour. Démo incluse sur notre cas d'usage RH.

Speakers: Wided Ahlem Touhami, Hamza Ben Marzouk, Ouarda Boumansour (Équipe AI Engineering, Wakam)

2) Du “large” au “small” : pourquoi les Small Language Models changent la donne Les LLMs sont impressionnants, mais coûteux et lourds à opérer et parfois disproportionnés par rapport aux besoins réels. Les Small Language Models proposent une alternative plus légère, adaptable dans de bonnes conditions, moins chère et beaucoup plus simple à déployer, tout en couvrant une large partie des cas d’usage opérationnels. Voyons ensemble comment et pourquoi “small” devient souvent le choix le plus pragmatique… et parfois le plus stratégique.

Speaker: Olivier Bergeret (Head of Data)

Lieu: Thiga, 23 rue Taitbout, 75009 Paris Accueil: 18:45 Talks: 19:00 puis Q&A Apéro: networking sur place Places limitées. Merci à Thiga pour l’accueil.

Network, learn, ask a question, meet other folks - these are all things that happen at user group events. These events are a really great opportunity to socialise in an informal learning experience.

Remember to tell your friends and the people you work with; make sure you register as soon as you can.

In-Person only event. Not being recorded.

17:45 – 18:00 Intro and updates 18:00 – 19:00 Shubhangi Goyal : Crafting LLM Applications: Design, Build, and Evaluate Large Language Models (LLMs) have revolutionised how applications are designed, offering capabilities such as natural language understanding, content generation, and conversational engagement. Building LLM-powered applications involves strategic design to leverage these capabilities effectively, focusing on context-awareness, domain-specific fine-tuning, and user-centred workflows. Integrating APIs, data pipelines, and feedback loops during development ensures that the application evolves and adapts to user needs, enhancing performance over time.

A critical aspect of LLM application development is response evaluation, which ensures outputs are accurate, coherent, and ethically aligned. Employing techniques like automated scoring, human-in-the-loop feedback, and real-world testing helps refine responses and maintain quality. Addressing challenges such as bias, and alignment with organizational goals ensures that LLM applications are not only functional but also trustworthy. This comprehensive approach to design, development, and evaluation allows organizations to build impactful LLM applications that deliver meaningful interactions across various use cases.

19:00 – 19:15 Break & Pizza 19:15 – 20:15 Atul Thakor : Going from 0 to 50k users in 5 months, An AI start up with zero marketing. This isn’t theory or productivity p*rn. If you want an insight into the rollercoaster world of startups, an example of rapid tech growth, and the technology behind it, then this is for you. No prior AI knowledge or technical background is needed – everything will be explained in plain English. This talk shares the journey of building an app to solve a problem for the speaker's own children, only to see it explode in popularity – growing locally, nationally, and then globally at speed. You'll learn about:

• Navigating Hyper-Growth: The unique challenges encountered when scaling unexpectedly fast.
• Why No One AI is Best: Understanding the limitations and strengths of different models.
• Giving Backend Developers Superpowers: Exploring 'vibe coding' and its implications.
• Why Multi-Agent is the Future: Understanding how to build a best-of-breed solution.

This session blends storytelling with practical, technical perspectives drawn from direct experience in building, scaling, and adapting AI technology in the real world.

I’m doing things a bit differently for this episode of Experiencing Data. For the first time on the show, I’m hosting a panel discussion. I’m joined by Thomson Reuters’s Simon Landry, Sumo Logic’s Greg Nudelman, and Google’s Paz Perez to chat about how we design user experiences that improve people’s lives and create business impact when we expose LLM capabilities to our users. 

With the rise of AI, there are a lot of opportunities for innovation, but there are also many challenges—and frankly, my feeling is that a lot of these capabilities right now are making things worse for users, not better. We’re looking at a range of topics such as the pros and cons of AI-first thinking, collaboration between UX designers and ML engineers, and the necessity of diversifying design teams when integrating AI and LLMs into b2b products. 

Highlights/ Skip to 

Thoughts on how the current state of LLMs implementations and its impact on user experience (1:51)  The problems that can come with the "AI-first" design philosophy (7:58)  Should a company's design resources be spent on go toward AI development? (17:20) How designers can navigate "fuzzy experiences” (21:28) Why you need to narrow and clearly define the problems you’re trying to solve when building LLMs products (27:35) Why diversity matters in your design and research teams when building LLMs (31:56)  Where you can find more from Paz, Greg, and Simon (40:43)

Quotes from Today’s Episode

“ [AI] will connect the dots. It will argue pro, it will argue against, it will create evidence supporting and refuting, so it’s really up to us to kind of drive this. If we understand the capabilities, then it is an almost limitless field of possibility. And these things are taught, and it’s a fundamentally different approach to how we build user interfaces. They’re no longer completely deterministic. They’re also extremely personalized to the point where it’s ridiculous.” - Greg Nudelman (12:47) “ To put an LLM into a product means that there’s a non-zero chance your user is going to have a [negative] experience and no longer be your customer. That is a giant reputational risk, and there’s also a financial cost associated with running these models. I think we need to take more of a service design lens when it comes to [designing our products with AI] and ask what is the thing somebody wants to do… not on my website, but in their lives? What brings them to my [product]? How can I imagine a different world that leverages these capabilities to help them do their job? Because what [designers] are competing against is [a customer workflow] that probably worked well enough.” - Simon Landry (15:41) “ When we go general availability (GA) with a product, that traditionally means [designers] have done all the research, got everything perfect, and it’s all great, right? Today, GA is a starting gun. We don’t know [if the product is working] unless we [seek out user feedback]. A massive research method is needed. [We need qualitative research] like sitting down with the customer and watching them use the product to really understand what is happening[…] but you also need to collect data. What are they typing in? What are they getting back? Is somebody who’s typing in this type of question always having a short interaction? Let’s dig into it with rapid, iterative testing and evaluation, so that we can update our model and then move forward. Launching a product these days means the starting guns have been fired. Put the research to work to figure out the next step.” - (23:29) Greg Nudelman “ I think that having diversity on your design team (i.e. gender, level of experience, etc.) is critical. We’ve already seen some terrible outcomes. Multiple examples where an LLM is crafting horrendous emails, introductions, and so on. This is exactly why UXers need to get involved [with building LLMs]. This is why diversity in UX and on your tech team that deals with AI is so valuable. Number one piece of advice: get some researchers. Number two: make sure your team is diverse.” - Greg Nudelman (32:39) “ It’s extremely important to have UX talks with researchers, content designers, and data teams. It’s important to understand what a user is trying to do, the context [of their decisions], and the intention. [Designers] need to help [the data team] understand the types of data and prompts being used to train models. Those things are better when they’re written and thought of by [designers] who understand where the user is coming from. [Design teams working with data teams] are getting much better results than the [teams] that are working in a vacuum.” - Paz Perez (35:19)

Links

Milly Barker’s LinkedIn post Greg Nudelman’s Value Matrix Article Greg Nudelman website  Paz Perez on Medium Paz Perez on LinkedIn Simon Landry LinkedIn

The Data Product Management In Action podcast, brought to you by Soda and executive producer Scott Hirleman, is a platform for data product management practitioners to share insights and experiences. 

In Season 01, Episode 17, host Nick Zervoudis ( Head of Product at CKDelta) talks to Grace Halim (Product Manager with Power Digital Marketing). In this episode Grace shares her career journey and highlights the importance of diverse experiences in shaping a successful product manager. She discusses the value of empathy, strong relationships with engineering and data teams, and the role of curiosity in asking the right questions. Listen to Grace as she shares her plans for a career break to explore new projects! 

About our host Nick Zervoudis: Nick is Head of Product at CKDelta, an AI software business within the CK Hutchison Holdings group. Nick oversees a portfolio of data products and works with sister companies to uncover new opportunities to innovate using data, analytics, and machine learning. Nick's career has revolved around data and advanced analytics from day one, having worked as an analyst, consultant, product manager, and instructor for startups, SMEs, and enterprises including PepsiCo, Sainsbury's, Lloyds Banking Group, IKEA, Capgemini Invent, BrainStation, QuantSpark, and Hg Capital. Nick is also the co-host of London's Data Product Management meetup, and speaks & writes regularly about data & AI product management. Connect with Nick on LinkedIn.  
About our guest Grace Halim: Grace is a seasoned product leader with a passion for building innovative products. With over 12 years of product management experience, Grace has honed her skills in leading high-performing product teams and delivering exceptional customer experiences. From crafting engaging data products to optimizing complex enterprise systems, Grace has a proven track record of success in the product management field. Grace is currently on a career break traveling around Australia in a caravan with her young family. Having been a product leader in the last two roles she held, Grace excelled in building and scaling product teams, fostering a culture of innovation, and driving business growth. Her teams' focus on customer focus and strategic thinking have been instrumental in delivering successful products that resonate with customers and drive bottom-line results. Beyond her corporate experience, Grace is an entrepreneur at heart. As co-founder of a data platform, she demonstrated her ability to turn a vision into a paying customer. Stay up to date with Grace’s adventure on LinkedIn. 

All views and opinions expressed are those of the individuals and do not necessarily reflect their employers or anyone else. 

Join the conversation on LinkedIn. 

Apply to be a guest or nominate someone that you know. 

Do you love what you're listening to? Please rate and review the podcast, and share it with fellow practitioners you know. Your support helps us reach more listeners and continue providing valuable insights! 

About the Event:

Join us for a 2-day virtual conference on March 6 & 7 to discover the latest services and features in Azure designed specifically for JavaScript developers. You'll hear directly from the experts behind the most sought-after cloud services for developers to learn cutting-edge cloud development techniques that can save you time and money, while providing your customers with the best experience possible.

Join us for our Cloud Skills Challenge at https://aka.ms/AzureJSDay-Challenge.

Who Should Attend:

This event is for everyone interested in learning about JavaScript in Azure and all the opportunities you can unlock, regardless of your level of experience with Azure or JavaScript!

Agenda:

DAY 1: BUILDING AN INTELLIGENT APP WITH JS: DEV TOOLS AND AI

9:00 - 9:10 Introduction and Setup (10m) - Natalia Venditto, Principal Product Owner JavaScript E2E DevEx; Dina Berry, Senior Content Developer Azure JavaScript DevEx

Welcome to the start of the day where we will go over the agenda and then get set up with forking the repo we will use for this event.

9:10 - 9:40 GitHub Copilot Can Do That? (30 m) - Burke Holland, Principal Cloud Advocate VS Code

It’s hard to go even a single day anymore without hearing the word “Copilot”. GitHub Copilot is the original Copilot and the most widely adopted AI tool in history. In this session, we’ll jump into GitHub Copilot and take a look at some of the astonishing things that it can do and how it can make your life as a developer exponentially easier and more enjoyable.

9:40 - 10:25 Building a versatile RAG Pattern chat bot with Azure OpenAI, LangChain (45m) - Wassim Chegham, Senior Software Engineer JavaScript Developer Advocacy; Natalia Venditto, Principal Product Owner JavaScript E2E DevEx; Lars Gyrup Brink Nielsen, Microsoft MVP

In this session we will walk you through the code of our popular JavaScript Azure OpenAI sample, from the backend services, to the frontend application, and even the schema that connects them seamlessly together: the Chat Application Protocol. Lars will also present the most cutting edge new features of Angular in its version 17, a favorite to build enterprise scale applications with!

10:25 - 10:45 LangChain.js + Azure: A Generative AI App Journey (20m) - Yohan Lasorsa, Senior JavaScript Developer Advocate

Discover the journey of building a generative AI application using LangChain.js and Azure. This talk will explore the development process from idea to production, focusing on a RAG-based approach for a Q&A system using YouTube video transcripts. We'll demonstrate how we built a local prototype using open-source models and Ollama, and its transition to Azure for production.

10:45 - 11:10 Extending Copilot for Microsoft 365 using JavaScript and TypeScript (25m) - Bob German, Principal Cloud Advocate Microsoft 365

You may have heard that Microsoft 365 now has an AI Copilot to help users do more within Microsoft 365. What you might not know is that you can extend Copilot to work with your business data and external content. In this session you’ll learn how to extend Copilot with plugins and Graph connectors written in JavaScript and TypeScript. We’ll examine the architecture, relevant Azure resources, and of course the code. All code will be made available so you can try it yourself. It’s easy – please join the session to get started!

11:10 – 11:30 Have a safe coffee chat with your documentation using Azure AI Services (20m) - Maya Shavin, Senior Software Engineer

Building a custom documentation assistant using AI has become achievable with the help of GPT, LangChain and other AI tools. But how can we control the content quality of the coffee chat made to our document assistant, from the user to the assistant’s response? What options do we have to enhance the content safety in our question-and-answer flow, while scaling our project to handle further scenarios? Join my talk and let’s find out.

11:30-11:40 Outro (10m)

DAY 2: BUILDING AN INTELLIGENT APP WITH JS: HOSTING AND INTEGRATIONS

9:00 - 9:10 Introduction (10m) – Natalia Venditto, Principal Product Owner JavaScript E2E DevEx,

Welcome to the start of day two where we will go over today’s agenda and recap yesterday’s event.

9:10 - 9:40 Crafting Future-proof Apps with JavaScript & Azure Cosmos DB (30m) - Sajeetharan Sinnathurai, Principal Product Manager Azure Cosmos DB

In this session, we'll discuss the developer experience of Cosmos DB with JavaScript, covering the latest additions to the SDK. Additionally, we'll explore Vercel integration for seamless deployment of JavaScript-based applications using templates.

9:40 - 10:00 Turn your database into GraphQL APIs with Azure Static Web Apps Database Connections (20m) - Thomas Gavin, Product Manager Azure Static Web Apps; Stacy Cashmore, MVP for Developer Technologies

Skip the boilerplate server code and use Static Web Apps Database Connections to directly access your database contents using a set of provided GraphQL APIs. In this session, we demo how you can quickly go from frontend to full-stack, saving results in a CosmosDB database using Database Connections and deploying to Azure Static Web Apps.

10:00 – 10:25 Build real-time web apps with Socket.IO and let Azure handle scalability, no more adapters (25m) - Ken Chen, Principal Software Eng Manager Azure Web PubSub and Azure SignalR; Kevin Guo, Senior Product manager Azure Web PubSub and Azure SignalR

Socket.IO is a popular open-source library among JavaScript developers for building real-time web apps. In this session, we are going to explore what we mean by “real-time” web apps and how Socket.IO library can help web developers build them. Also, we discuss a common challenge among Socket.IO developers – scaling out to multiple Socket.IO servers. Through a quick demo, we showcase how easy it is to leverage the recently introduced support for Socket.IO on Azure to offload scalability issue to a cloud service.

10:25 - 11:10 Playwright in Action: From Setup to Best Practices (45m) - Max Schmitt, Software Engineer Playwright; Stefan Judis, Playwright Ambassador

Dive into the essentials of end-to-end testing with Playwright in this engaging 45-minute session. A Playwright core contributor will guide you through a hands-on demo, demonstrating how to efficiently set up, execute automated tests and debug them in GitHub Actions. After that a Playwright ambassador will share the best practices and tips to optimize your testing workflow.

11:10 – 11:20 Outro (10m)

** This is a great chance to start or advance your journey towards improving your developer productivity and innovation. Join us for exciting sessions with insights, useful tips, and interactive discussions that will help you unlock your full potential as a JavaScript developer. We can't wait to see you there on March 6 & 7! **

Join us as we interview my bootcamp student Erin Shina who recently pivoted her career from music therapy to a financial analyst role at Fortune 50 company Humana.

⁠🏫 Check out my 10-week data analytics bootcamp⁠

📊 Come to my next free “How to Land Your First Data Job” training

Timestamps:

(2:49) - Hybrid work is the new norm, combining office and remote setups. 💼🏠

(6:00) - A supportive boss is crucial in advancing your data career. 👩‍💼🚀

(9:00) - Music therapy offers a fulfilling career path in healthcare. 🎵⚕️

(10:11) - Discover how the Data Analytics Accelerator helped Erin secure a data job. 📊🔍

(14:16) - Showcase your project effectively to increase your interview chances. 📂🎯

(17:25) - Put extra effort into crafting an impactful resume. 📄💪

(21:41) - Your current network could become a valuable referral for your next job opportunity. 🌐🔗

Connect with Avery:

📺 Subscribe on YouTube

🎙Listen to My Podcast

👔 Connect with me on LinkedIn

📸 Instagram

🎵 TikTok

Mentioned in this episode: Join the last cohort of 2025! The LAST cohort of The Data Analytics Accelerator for 2025 kicks off on Monday, December 8th and enrollment is officially open!

To celebrate the end of the year, we’re running a special End-of-Year Sale, where you’ll get: ✅ A discount on your enrollment 🎁 6 bonus gifts, including job listings, interview prep, AI tools + more

If your goal is to land a data job in 2026, this is your chance to get ahead of the competition and start strong.

👉 Join the December Cohort & Claim Your Bonuses: https://DataCareerJumpstart.com/daa https://www.datacareerjumpstart.com/daa

Today I’m discussing something we’ve been talking about a lot on the podcast recently - the definition of a “data product.” While my definition is still a work in progress, I think it’s worth putting out into the world at this point to get more feedback. In addition to sharing my definition of data products (as defined the “producty way”), on today’s episode definition, I also discuss some of the non-technical skills that data product managers (DPMs) in the ML and AI space need if they want to achieve good user adoption of their solutions. I’ll also share my thoughts on whether data scientists can make good data product managers, what a DPM can do to better understand your users and stakeholders, and how product and UX design factors into this role. 

Highlights/ Skip to:

I introduce my reasons for sharing my definition of a data product (0:46) My definition of data product (7:26) Thinking the “producty” way (8:14) My thoughts on necessary skills for data PMs (in particular, AI & machine learning product management) (12:21) How data scientists can become good data product managers (DPMs) by taking off the data science hat (13:42) Understanding the role of UX design within the context of DPM (16:37) Crafting your sales and marketing strategies to emphasize the value of your product to the people who can use or purchase it (23:07) How to build a team that will help you increase adoption of your data product (30:01) How to build relationships with stakeholders/customers that allow you to find the right solutions for them (33:47) Letting go of a technical identity to develop a new identity as a DPM who can lead a team to build a product that actually gets used (36:32)

Quotes from Today’s Episode “This is what’s missing in some of the other definitions that I see around data products  [...] they’re not talking about it from the customer of the data product lens. And that orientation sums up all of the work that I’m doing and trying to get you to do as well, which is to put the people at the center of the work that you’re doing and not the data science, engineering, tech, or design. I want you to put the people at the center.” (6:12) “A data product is a data-driven, end-to-end, human-in-the-loop decision support solution that’s so valuable, users would potentially pay to use it.” (7:26) “I want to plunge all the way in and say, ‘if you want to do this kind of work, then you need to be thinking the product-y way.’ And this means inherently letting go of some of the data science-y way of thinking and the data-first kinds of ways of thinking.” (11:46) “I’ve read in a few places that data scientists don’t make for good data product managers. [While it may be true that they’re more introverted,] I don’t think that necessarily means that there’s an inherent problem with data scientists becoming good data product managers. I think the main challenge will be—and this is the same thing for almost any career transitioning into product management—is knowing when to let go of your former identity and wear the right hat at the right time.” (14:24) “Make better things for people that will improve their life and their outcomes and the business value will follow if you’ve properly aligned those two things together.” (17:21) “The big message here is this: there is always a design and experience, whether it is an API, or a platform, a dashboard, a full application, etc. Since there are no null design choices, how much are you going to intentionally shape that UX, or just pray that it comes out good on the other end? Prayer is not really a reliable strategy.  If you want to routinely do this work right, you need to put intention behind it.” (22:33)  “Relationship building is a must, and this is where applying user experience research can be very useful—not just for users, but also with stakeholders. It’s learning how to ask really good questions and learning the feelings, emotions, and reasons why people ask your team to build the thing that they’ve asked for. Learning how to dig into that is really important.” (26:26)

Links Designing for Analytics Community Work With Me Email Record a question