In today’s episode, I’m going to perhaps work myself out of some consulting engagements, but hey, that’s ok! True consulting is about service—not PPT decks with strategies and tiers of people attached to rate cards. Specifically today, I decided to reframe a topic and approach it from the opposite/negative side. So, instead of telling you when the right time is to get UX design help for your enterprise SAAS analytics or AI product(s), today I’m going to tell you when you should NOT get help! 

Reframing this was really fun and made me think a lot as I recorded the episode. Some of these reasons aren’t necessarily representative of what I believe, but rather what I’ve heard from clients and prospects over 25 years—what they believe. For each of these, I’m also giving a counterargument, so hopefully, you get both sides of the coin. 

Finally, analytical thinkers, especially data product managers it seems, often want to quantify all forms of value they produce in hard monetary units—and so in this episode, I’m also going to talk about other forms of value that products can create that are worth paying for—and how mushy things like “feelings” might just come into play ;-)  Ready?

Highlights/ Skip to:

(1:52) Going for short, easy wins (4:29) When you think you have good design sense/taste  (7:09) The impending changes coming with GenAI (11:27) Concerns about "dumbing down" or oversimplifying technical analytics solutions that need to be powerful and flexible (15:36) Agile and process FTW? (18:59) UX design for and with platform products (21:14) The risk of involving designers who don’t understand data, analytics, AI, or your complex domain considerations  (30:09) Designing after the ML models have been trained—and it’s too late to go back  (34:59) Not tapping professional design help when your user base is small , and you have routine access and exposure to them   (40:01) Explaining the value of UX design investments to your stakeholders when you don’t 100% control the budget or decisions 

Quotes from Today’s Episode “It is true that most impactful design often creates more product and engineering work because humans are messy. While there sometimes are these magic, small GUI-type changes that have big impact downstream, the big picture value of UX can be lost if you’re simply assigning low-level GUI improvement tasks and hoping to see a big product win. It always comes back to the game you’re playing inside your team: are you working to produce UX and business outcomes or shipping outputs on time? ” (3:18) “If you’re building something that needs to generate revenue, there has to be a sense of trust and belief in the solution. We’ve all seen the challenges of this with LLMs. [when] you’re unable to get it to respond in a way that makes you feel confident that it understood the query to begin with. And then you start to have all these questions about, ‘Is the answer not in there,’ or ‘Am I not prompting it correctly?’ If you think that most of this is just an technical data science problem, then don’t bother to invest in UX design work… ” (9:52) “Design is about, at a minimum, making it useful and usable, if not delightful. In order to do that, we need to understand the people that are going to use it. What would an improvement to this person’s life look like? Simplifying and dumbing things down is not always the answer. There are tools and solutions that need to be complex, flexible, and/or provide a lot of power – especially in an enterprise context. Working with a designer who solely insists on simplifying everything at all costs regardless of your stated business outcome goals is a red flag—and a reason not to invest in UX design—at least with them!“ (12:28)“I think what an analytics product manager [or] an AI product manager needs to accept is there are other ways to measure the value of UX design’s contribution to your product and to your organization. Let’s say that you have a mission-critical internal data product, it’s used by the most senior executives in the organization, and you and your team made their day, or their month, or their quarter. You saved their job. You made them feel like a hero. What is the value  of giving them that experience and making them feel like those things… What is that worth when a key customer or colleague feels like you have their back with this solution you created? Ideas that spread, win, and if these people are spreading your idea, your product, or your solution… there’s a lot of value in that.” (43:33)

“Let’s think about value in non-financial terms. Terms like feelings. We buy insurance all the time. We’re spending money on something that most likely will have zero economic value this year because we’re actually trying not to have to file claims. Yet this industry does very well because the feeling of security matters. That feeling is worth something to a lot of people. The value of feeling secure is something greater than whatever the cost of the insurance plan. If your solution can build feelings of confidence and security, what is that worth? Does “hard to measure precisely” necessarily mean “low value?”  (47:26)

Last week I talked about how good you have to be at your job. Yesterday's OpenAI announcement of it's "reasoning" model, o1, got me thinking about how good AI needs to be to do our jobs.

As AI becomes more accessible, a growing question is: should machine learning experts always be the ones training models, or is there a better way to leverage other subject matter experts in the business who know the use-case best? What if getting started building AI apps required no coding skills? As businesses look to implement AI at scale, what part can no-code AI apps play in getting projects off the ground, and how feasible are smaller, tailored solutions for  department specific use-cases? Birago Jones is the CEO at Pienso. Pienso is an AI platform that empowers subject matter experts in various enterprises, such as business analysts, to create and fine-tune AI models without coding skills. Prior to Pienso, Birago was a Venture Partner at Indicator Ventures and a Research Assistant at MIT Media Lab where he also founded the Media Lab Alumni Association. Karthik Dinakar is a computer scientist specializing in machine learning, natural language processing, and human-computer interaction. He is the Chief Technology Officer and co-founder at Pienso. Prior to founding Pienso, Karthik held positions at Microsoft and Deutsche Bank. Karthik holds a doctoral degree from MIT in Machine Learning. In the episode, Richie, Birago and Karthik explore why no-code AI apps are becoming more prominent, uses-cases of no-code AI apps, the steps involved in creating an LLM, the benefits of small tailored models, how no-code can impact workflows, cost in AI projects, AI interfaces and the rise of the chat interface, privacy and customization, excitement about the future of AI, and much more.  Links Mentioned in the Show: PiensoGoogle Gemini for BusinessConnect with Birago and KarthikAndreesen Horowitz Report: Navigating the High Cost of AI ComputeCourse: Artificial Intelligence (AI) StrategyRelated Episode: Designing AI Applications with Robb Wilson, Co-Founder & CEO at Onereach.aiRewatch sessions from RADAR: AI Edition New to DataCamp? Learn on the go using the DataCamp mobile appEmpower your business with world-class data and AI skills with DataCamp for business

Great data presentations tell a story. Learn how to organize, visualize, and present data using Python, generative AI, and the cutting-edge Altair data visualization toolkit. Take the fast track to amazing data presentations! Data Storytelling with Altair and AI introduces a stack of useful tools and tried-and-tested methodologies that will rapidly increase your productivity, streamline the visualization process, and leave your audience inspired. In Data Storytelling with Altair and AI you’ll discover: Using Python Altair for data visualization Using Generative AI tools for data storytelling The main concepts of data storytelling Building data stories with the DIKW pyramid approach Transforming raw data into a data story Data Storytelling with Altair and AI teaches you how to turn raw data into effective, insightful data stories. You’ll learn exactly what goes into an effective data story, then combine your Python data skills with the Altair library and AI tools to rapidly create amazing visualizations. Your bosses and decision-makers will love your new presentations—and you’ll love how quick Generative AI makes the whole process! About the Technology Every dataset tells a story. After you’ve cleaned, crunched, and organized the raw data, it’s your job to share its story in a way that connects with your audience. Python’s Altair data visualization library, combined with generative AI tools like Copilot and ChatGPT, provide an amazing toolbox for transforming numbers, code, text, and graphics into intuitive data presentations. About the Book Data Storytelling with Altair and AI teaches you how to build enhanced data visualizations using these tools. The book uses hands-on examples to build powerful narratives that can inform, inspire, and motivate. It covers the Altair data visualization library, along with AI techniques like generating text with ChatGPT, creating images with DALL-E, and Python coding with Copilot. You’ll learn by practicing with each interesting data story, from tourist arrivals in Portugal to population growth in the USA to fake news, salmon aquaculture, and more. What's Inside The Data-Information-Knowledge-Wisdom (DIKW) pyramid Publish data stories using Streamlit, Tableau, and Comet Vega and Vega-Lite visualization grammar About the Reader For data analysts and data scientists experienced with Python. No previous knowledge of Altair or Generative AI required. About the Author Angelica Lo Duca is a researcher at the Institute of Informatics and Telematics of the National Research Council, Italy. The technical editor on this book was Ninoslav Cerkez. Quotes This book’s step-by-step approach, illustrated through real-world examples, makes complex data accessible and actionable. - Alexey Grigorev, DataTalks.Club A clear and concise guide to data storytelling. Highly recommended. - Andrew Madson, Insights x Design Data storytelling in a way that anyone can do! This book feels ahead of its time. - Avery Smith, Data Career Jumpstart Excellent hands-on exercises that combine two of my favorite tools: AI and the Altair library. - Jose Berengueres, Author of DataViz and Storytelling

LLMs have opened up new avenues in NLP with their possible applications, but evaluating their output introduces a new set of challenges. In this talk, we discuss how the evaluation of LLMs differs from the evaluation of classic ML-based solutions and how we tackle the challenges.

Abstract: The vast availability of unstructured data presents a significant opportunity for social sciences, yet there is a pressing need for better tools and infrastructure to access and utilize this data effectively. This talk will highlight how the Business and Economic Research Data Infrastructure Program BERD@NFDI is addressing these needs, showcasing achievements and inviting further collaboration within the European social science community. Simultaneously, the fields of Natural Language Processing (NLP) and Large Language Models (LLMs) require high-quality training data. Social scientists have been collecting valuable data for decades, which can serve as essential benchmarks for advancing NLP and LLM research. By embracing open science, we can bridge the gap between social science and computational research, making this data more accessible and fostering collaboration across disciplines.

Lot’s of AI use-cases can start with big ideas and exciting possibilities, but turning those ideas into real results is where the challenge lies. How do you take a powerful model and make it work effectively in a specific business context? What steps are necessary to fine-tune and optimize your AI tools to deliver both performance and cost efficiency? And as AI continues to evolve, how do you stay ahead of the curve while ensuring that your solutions are scalable and sustainable?  Lin Qiao is the CEO and Co-Founder of Fireworks AI. She previously worked at Meta as a Senior Director of Engineering and as head of Meta's PyTorch, served as a Tech Lead at Linkedin, and worked as a Researcher and Software Engineer at IBM.  In the episode, Richie and Lin explore generative AI use cases, getting AI into products, foundational models, the effort required and benefits of fine-tuning models, trade-offs between models sizes, use cases for smaller models, cost-effective AI deployment, the infrastructure and team required for AI product development, metrics for AI success, open vs closed-source models, excitement for the future of AI development and much more.  Links Mentioned in the Show: Fireworks.aiHugging Face - Preference Tuning LLMs with Direct Preference Optimization MethodsConnect with LinCourse - Artificial Intelligence (AI) StrategyRelated Episode: Creating Custom LLMs with Vincent Granville, Founder, CEO & Chief Al Scientist at GenAltechLab.comRewatch sessions from RADAR: AI Edition New to DataCamp? Learn on the go using the DataCamp mobile appEmpower your business with world-class data and AI skills with DataCamp for business

“Last week was a great year in GenAI,” jokes Mark Ramsey—and it’s a great philosophy to have as LLM tools especially continue to evolve at such a rapid rate. This week, you’ll get to hear my fun and insightful chat with Mark from Ramsey International about the world of large language models (LLMs) and how we make useful UXs out of them in the enterprise. 

Mark shared some fascinating insights about using a company’s website information (data) as a place to pilot a LLM project, avoiding privacy landmines, and how re-ranking of models leads to better LLM response accuracy. We also talked about the importance of real human testing to ensure LLM chatbots and AI tools truly delight users. From amusing anecdotes about the spinning beach ball on macOS to envisioning a future where AI-driven chat interfaces outshine traditional BI tools, this episode is packed with forward-looking ideas and a touch of humor.

Highlights/ Skip to:

(0:50) Why is the world of GenAI evolving so fast? (4:20) How Mark thinks about UX in an LLM application (8:11) How Mark defines “Specialized GenAI?” (12:42) Mark’s consulting work with GenAI / LLMs these days (17:29) How GenAI can help the healthcare industry (30:23) Uncovering users’ true feelings about LLM applications (35:02) Are UIs moving backwards as models progress forward? (40:53) How will GenAI impact data and analytics teams? (44:51) Will LLMs be able to consistently leverage RAG and produce proper SQL? (51:04) Where can find more from Mark and Ramsey International

Quotes from Today’s Episode “With [GenAI], we have a solution that we’ve built to try to help organizations, and build workflows. We have a workflow that we can run and ask the same question [to a variety of GenAI models] and see how similar the answers are. Depending on the complexity of the question, you can see a lot of variability between the models… [and] we can also run the same question against the different versions of the model and see how it’s improved. Folks want a human-like experience interacting with these models.. [and] if the model can start responding in just a few seconds, that gives you much more of a conversational type of experience.” - Mark Ramsey (2:38) “[People] don’t understand when you interact [with GenAI tools] and it brings tokens back in that streaming fashion, you’re actually seeing inside the brain of the model. Every token it produces is then displayed on the screen, and it gives you that typewriter experience back in the day. If someone has to wait, and all you’re seeing is a logo spinning, from a UX experience standpoint… people feel like the model is much faster if it just starts to produce those results in that streaming fashion. I think in a design, it’s extremely important to take advantage of that [...] as opposed to waiting to the end and delivering the results some models support that, and other models don’t.”- Mark Ramsey (4:35) "All of the data that’s on the website is public information. We’ve done work with several organizations on quickly taking the data that’s on their website, packaging it up into a vector database, and making that be the source for questions that their customers can ask. [Organizations] publish a lot of information on their websites, but people really struggle to get to it. We’ve seen a lot of interest in vectorizing website data, making it available, and having a chat interface for the customer. The customer can ask questions, and it will take them directly to the answer, and then they can use the website as the source information.” - Mark Ramsey (14:04) “I’m not skeptical at all. I’ve changed much of my [AI chatbot searches] to Perplexity, and I think it’s doing a pretty fantastic job overall in terms of quality. It’s returning an answer with citations, so you have a sense of where it’s sourcing the information from. I think it’s important from a user experience perspective. This is a replacement for broken search, as I really don’t want to read all the web pages and PDFs you have that might be about my chiropractic care query to answer my actual [healthcare] question.” - Brian O’Neill (19:22)

“We’ve all had great experience with customer service, and we’ve all had situations where the customer service was quite poor, and we’re going to have that same thing as we begin to [release more] chatbots. We need to make sure we try to alleviate having those bad experiences, and have an exit. If someone is running into a situation where they’d rather talk to a live person, have that ability to route them to someone else. That’s why the robustness of the model is extremely important in the implementation… and right now, organizations like OpenAI and Anthropic are significantly better at that [human-like] experience.” - Mark Ramsey (23:46) "There’s two aspects of these models: the training aspect and then using the model to answer questions. I recommend to organizations to always augment their content and don’t just use the training data. You’ll still get that human-like experience that’s built into the model, but you’ll eliminate the hallucinations. If you have a model that has been set up correctly, you shouldn’t have to ask questions in a funky way to get answers.” - Mark Ramsey (39:11) “People need to understand GenAI is not a predictive algorithm. It is not able to run predictions, it struggles with some math, so that is not the focus for these models. What’s interesting is that you can use the model as a step to get you [the answers]. A lot of the models now support functions… when you ask a question about something that is in a database, it actually uses its knowledge about the schema of the database. It can build the query, run the query to get the data back, and then once it has the data, it can reformat the data into something that is a good response back." - Mark Ramsey (42:02)

Links Mark on LinkedIn Ramsey International Email: mark [at] ramsey.international Ramsey International's YouTube Channel

Vinoo Ganesh is an open source enthusiast and contributor, and a data and ML engineer. We chat about strong open source communities, LLMs and AI, and much more.

In the age of language models, search engines are undergoing a revolution. Search technology is moving from traditional lexical methods to advanced semantic models, offering a more intuitive user experience. In this talk, we will look at the impact of this transition, and consider possible future directions of search technology.

Large language models (LLMs) and generative AI are rapidly changing the healthcare industry. These technologies have the potential to revolutionize healthcare by improving the efficiency, accuracy, and personalization of care. This practical book shows healthcare leaders, researchers, data scientists, and AI engineers the potential of LLMs and generative AI today and in the future, using storytelling and illustrative use cases in healthcare. Authors Kerrie Holley, former Google healthcare professionals, guide you through the transformative potential of large language models (LLMs) and generative AI in healthcare. From personalized patient care and clinical decision support to drug discovery and public health applications, this comprehensive exploration covers real-world uses and future possibilities of LLMs and generative AI in healthcare. With this book, you will: Understand the promise and challenges of LLMs in healthcare Learn the inner workings of LLMs and generative AI Explore automation of healthcare use cases for improved operations and patient care using LLMs Dive into patient experiences and clinical decision-making using generative AI Review future applications in pharmaceutical R&D, public health, and genomics Understand ethical considerations and responsible development of LLMs in healthcare "The authors illustrate generative's impact on drug development, presenting real-world examples of its ability to accelerate processes and improve outcomes across the pharmaceutical industry." --Harsh Pandey, VP, Data Analytics & Business Insights, Medidata-Dassault Kerrie Holley is a retired Google tech executive, IBM Fellow, and VP/CTO at Cisco. Holley's extensive experience includes serving as the first Technology Fellow at United Health Group (UHG), Optum, where he focused on advancing and applying AI, deep learning, and natural language processing in healthcare. Manish Mathur brings over two decades of expertise at the crossroads of healthcare and technology. A former executive at Google and Johnson & Johnson, he now serves as an independent consultant and advisor. He guides payers, providers, and life sciences companies in crafting cutting-edge healthcare solutions.

Perhaps the biggest complaint about generative AI is hallucination. If the text you want to generate involves facts, for example, a chatbot that answers questions, then hallucination is a problem. The solution to this is to make use of a technique called retrieval augmented generation, where you store facts in a vector database and retrieve the most appropriate ones to send to the large language model to help it give accurate responses. So, what goes into building vector databases and how do they improve LLM performance so much? Ram Sriharsha is currently the CTO at Pinecone. Before this role, he was the Director of Engineering at Pinecone and previously served as Vice President of Engineering at Splunk. He also worked as a Product Manager at Databricks. With a long history in the software development industry, Ram has held positions as an architect, lead product developer, and senior software engineer at various companies. Ram is also a long time contributor to Apache Spark.  In the episode, Richie and Ram explore common use-cases for vector databases, RAG in chatbots, steps to create a chatbot, static vs dynamic data, testing chatbot success, handling dynamic data, choosing language models, knowledge graphs, implementing vector databases, innovations in vector data bases, the future of LLMs and much more.  Links Mentioned in the Show: PineconeWebinar - Charting the Path: What the Future Holds for Generative AICourse - Vector Databases for Embeddings with PineconeRelated Episode: The Power of Vector Databases and Semantic Search with Elan Dekel, VP of Product at PineconeRewatch sessions from RADAR: AI Edition New to DataCamp? Learn on the go using the DataCamp mobile app Empower your business with world-class data and AI skills with DataCamp for business