talk-data.com talk-data.com

Filter by Source

Select conferences and events

Showing 16 results

Activities & events

Title & Speakers Event

Overview When building machine learning and data applications, a significant portion of your time will be dedicated to data wrangling - from content extraction and filtering out problematic and low quality data. In this hands-on session we will explore Data Prep Kit - an open source toolkit, designed to streamline these essential tasks. Attendees will learn first hand how to use the Data Prep Kit to improve overall data quality such as removing spam and low quality documents, removing HAP (Hate Abuse Profanity) speech, removing PII (Personally Identifiable Information) data, thus leading to higher quality dataset.

Description Join us for an interactive, hands-on session where you will learn to clean up data and prepare high quality datasets.

In this workshop we will do the following:

  • Extract content from various documents (PDFs, HTML)
  • cleanup and remove markups
  • Detect and remove SPAM content
  • Score and remove low-quality documents
  • Identify and remove PII data
  • Detect and remove HAP (Hate Abuse Profanity) speech from documents

More about Data Prep Kit : https://github.com/IBM/data-prep-kit

What do you need to participate in this workshop?

  • Comfortable in python programming language
  • We will run the workshop code using Google Collab (free) - no other setup is needed!

Session Type Workshop (hands-on)

Audience LLM app developers, data scientists, data engineers

Technical Level Beginner - Intermediate

Prerequisites

  • Comfortable in python programming language
  • We will run the workshop using Google Collab (free to use) - no other setup is needed!

Duration 60 mins

Industry Cross industry

Speaker Bio https://sujee.dev/bio

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] Workshop: Preparing High Quality Datasets with Data Prep Kit

Overview When building machine learning and data applications, a significant portion of your time will be dedicated to data wrangling - from content extraction and filtering out problematic and low quality data. In this hands-on session we will explore Data Prep Kit - an open source toolkit, designed to streamline these essential tasks. Attendees will learn first hand how to use the Data Prep Kit to improve overall data quality such as removing spam and low quality documents, removing HAP (Hate Abuse Profanity) speech, removing PII (Personally Identifiable Information) data, thus leading to higher quality dataset.

Description Join us for an interactive, hands-on session where you will learn to clean up data and prepare high quality datasets.

In this workshop we will do the following:

  • Extract content from various documents (PDFs, HTML)
  • cleanup and remove markups
  • Detect and remove SPAM content
  • Score and remove low-quality documents
  • Identify and remove PII data
  • Detect and remove HAP (Hate Abuse Profanity) speech from documents

More about Data Prep Kit : https://github.com/IBM/data-prep-kit

What do you need to participate in this workshop?

  • Comfortable in python programming language
  • We will run the workshop code using Google Collab (free) - no other setup is needed!

Session Type Workshop (hands-on)

Audience LLM app developers, data scientists, data engineers

Technical Level Beginner - Intermediate

Prerequisites

  • Comfortable in python programming language
  • We will run the workshop using Google Collab (free to use) - no other setup is needed!

Duration 60 mins

Industry Cross industry

Speaker Bio https://sujee.dev/bio

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] Workshop: Preparing High Quality Datasets with Data Prep Kit

Overview When building machine learning and data applications, a significant portion of your time will be dedicated to data wrangling - from content extraction and filtering out problematic and low quality data. In this hands-on session we will explore Data Prep Kit - an open source toolkit, designed to streamline these essential tasks. Attendees will learn first hand how to use the Data Prep Kit to improve overall data quality such as removing spam and low quality documents, removing HAP (Hate Abuse Profanity) speech, removing PII (Personally Identifiable Information) data, thus leading to higher quality dataset.

Description Join us for an interactive, hands-on session where you will learn to clean up data and prepare high quality datasets.

In this workshop we will do the following:

  • Extract content from various documents (PDFs, HTML)
  • cleanup and remove markups
  • Detect and remove SPAM content
  • Score and remove low-quality documents
  • Identify and remove PII data
  • Detect and remove HAP (Hate Abuse Profanity) speech from documents

More about Data Prep Kit : https://github.com/IBM/data-prep-kit

What do you need to participate in this workshop?

  • Comfortable in python programming language
  • We will run the workshop code using Google Collab (free) - no other setup is needed!

Session Type Workshop (hands-on)

Audience LLM app developers, data scientists, data engineers

Technical Level Beginner - Intermediate

Prerequisites

  • Comfortable in python programming language
  • We will run the workshop using Google Collab (free to use) - no other setup is needed!

Duration 60 mins

Industry Cross industry

Speaker Bio https://sujee.dev/bio

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] Workshop: Preparing High Quality Datasets with Data Prep Kit

Overview When building machine learning and data applications, a significant portion of your time will be dedicated to data wrangling - from content extraction and filtering out problematic and low quality data. In this hands-on session we will explore Data Prep Kit - an open source toolkit, designed to streamline these essential tasks. Attendees will learn first hand how to use the Data Prep Kit to improve overall data quality such as removing spam and low quality documents, removing HAP (Hate Abuse Profanity) speech, removing PII (Personally Identifiable Information) data, thus leading to higher quality dataset.

Description Join us for an interactive, hands-on session where you will learn to clean up data and prepare high quality datasets.

In this workshop we will do the following:

  • Extract content from various documents (PDFs, HTML)
  • cleanup and remove markups
  • Detect and remove SPAM content
  • Score and remove low-quality documents
  • Identify and remove PII data
  • Detect and remove HAP (Hate Abuse Profanity) speech from documents

More about Data Prep Kit : https://github.com/IBM/data-prep-kit

What do you need to participate in this workshop?

  • Comfortable in python programming language
  • We will run the workshop code using Google Collab (free) - no other setup is needed!

Session Type Workshop (hands-on)

Audience LLM app developers, data scientists, data engineers

Technical Level Beginner - Intermediate

Prerequisites

  • Comfortable in python programming language
  • We will run the workshop using Google Collab (free to use) - no other setup is needed!

Duration 60 mins

Industry Cross industry

Speaker Bio https://sujee.dev/bio

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] Workshop: Preparing High Quality Datasets with Data Prep Kit

Hands-on session to explore Data Prep Kit and accelerate data preparation for building robust LLM applications. Topics include getting started with Data Prep Kit, extracting content from PDFs, DOCX, and HTML, cleanup of excess markup, detecting/removing duplicate documents, and removing low-quality and spam documents. Attendees should be comfortable with Python; workshop code will run in Google Colab.

data prep kit Python google colab
[AI Alliance] Workshop: Hands-on with Data Prep Kit

Hands-on session to explore Data Prep Kit and how to accelerate data preparation for LLM applications. The workshop covers getting started with Data Prep Kit, extracting content from PDFs, DOCX, and HTML, cleaning markup, deduplicating content, and detecting/removing low-quality or spam documents.

data prep kit Python google colab
[AI Alliance] Workshop: Hands-on with Data Prep Kit
Data Prep Kit Workshop 2025-03-20 · 16:00

Hands-on workshop to explore IBM Data Prep Kit for data preparation, including getting started, extracting content from PDFs, DOCX, and HTML, cleaning markup, deduplicating data, and removing low-quality or spam documents. The session will be run in Google Colab and is suitable for LLM app developers, data scientists, and data engineers. Prerequisites: comfortable with Python.

Python google colab data prep kit
[AI Alliance] Workshop: Hands-on with Data Prep Kit

Overview When building machine learning and data applications, a significant portion of your time will be dedicated to data wrangling - from content extraction and cleaning to de-duplication and filtering out problematic data. In this hands-on session we will explore Data Prep Kit - an open source toolkit, designed to streamline these essential tasks. Attendees will learn first hand how to use the Data Prep Kit to accelerate data preparation, improve overall data quality, and enhance the efficiency of building robust LLM applications.

Description Data Prep Kit is a comprehensive Python library that democratizes and accelerates data preparation by providing out-of-the-box solutions for common tasks. Engineered to scale from a single laptop to large cloud clusters, it has been successfully used to process terabytes of data for training IBM Granite Large Language Models (LLMs).

Data Prep Kit offers a robust feature set including duplicate elimination, advanced document and code handling, language detection (for both spoken and programming languages), removal of personally identifiable information (PII), as well as spam, hate speech, and malware detection.

More about Data Prep Kit : https://github.com/IBM/data-prep-kit

Join us for this hands-on session to explore how to use Data Prep Kit to accelerate data preparation, enhance data quality.

In this workshop we will do the following:

  • getting started with Data Prep Kit
  • Extract content from various documents (PDFs, DOCX, HTML)
  • Cleanup documents by removing excess markup
  • Detect and remove duplicate documents
  • Detect and remove low quality and spam documents

What do you need to participate in this workshop?

  • Comfortable in python programming language
  • We will run the workshop code using Google Collab (free) - no other setup is needed!

Session Type Hands-on workshop

Audience LLM app developers, data scientists, data engineers

Technical Level Beginner - Intermediate

Prerequisites

  • Comfortable in python programming language
  • We will run the workshop using Google Collab (free) - no other setup is needed!

Duration 60 mins

Industry Cross industry

Speaker Bio https://sujee.dev/bio

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] Workshop: Hands-on with Data Prep Kit

Overview When building machine learning and data applications, a significant portion of your time will be dedicated to data wrangling - from content extraction and cleaning to de-duplication and filtering out problematic data. In this hands-on session we will explore Data Prep Kit - an open source toolkit, designed to streamline these essential tasks. Attendees will learn first hand how to use the Data Prep Kit to accelerate data preparation, improve overall data quality, and enhance the efficiency of building robust LLM applications.

Description Data Prep Kit is a comprehensive Python library that democratizes and accelerates data preparation by providing out-of-the-box solutions for common tasks. Engineered to scale from a single laptop to large cloud clusters, it has been successfully used to process terabytes of data for training IBM Granite Large Language Models (LLMs).

Data Prep Kit offers a robust feature set including duplicate elimination, advanced document and code handling, language detection (for both spoken and programming languages), removal of personally identifiable information (PII), as well as spam, hate speech, and malware detection.

More about Data Prep Kit : https://github.com/IBM/data-prep-kit

Join us for this hands-on session to explore how to use Data Prep Kit to accelerate data preparation, enhance data quality.

In this workshop we will do the following:

  • getting started with Data Prep Kit
  • Extract content from various documents (PDFs, DOCX, HTML)
  • Cleanup documents by removing excess markup
  • Detect and remove duplicate documents
  • Detect and remove low quality and spam documents

What do you need to participate in this workshop?

  • Comfortable in python programming language
  • We will run the workshop code using Google Collab (free) - no other setup is needed!

Session Type Hands-on workshop

Audience LLM app developers, data scientists, data engineers

Technical Level Beginner - Intermediate

Prerequisites

  • Comfortable in python programming language
  • We will run the workshop using Google Collab (free) - no other setup is needed!

Duration 60 mins

Industry Cross industry

Speaker Bio https://sujee.dev/bio

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] Workshop: Hands-on with Data Prep Kit

Overview When building machine learning and data applications, a significant portion of your time will be dedicated to data wrangling - from content extraction and cleaning to de-duplication and filtering out problematic data. In this hands-on session we will explore Data Prep Kit - an open source toolkit, designed to streamline these essential tasks. Attendees will learn first hand how to use the Data Prep Kit to accelerate data preparation, improve overall data quality, and enhance the efficiency of building robust LLM applications.

Description Data Prep Kit is a comprehensive Python library that democratizes and accelerates data preparation by providing out-of-the-box solutions for common tasks. Engineered to scale from a single laptop to large cloud clusters, it has been successfully used to process terabytes of data for training IBM Granite Large Language Models (LLMs).

Data Prep Kit offers a robust feature set including duplicate elimination, advanced document and code handling, language detection (for both spoken and programming languages), removal of personally identifiable information (PII), as well as spam, hate speech, and malware detection.

More about Data Prep Kit : https://github.com/IBM/data-prep-kit

Join us for this hands-on session to explore how to use Data Prep Kit to accelerate data preparation, enhance data quality.

In this workshop we will do the following:

  • getting started with Data Prep Kit
  • Extract content from various documents (PDFs, DOCX, HTML)
  • Cleanup documents by removing excess markup
  • Detect and remove duplicate documents
  • Detect and remove low quality and spam documents

What do you need to participate in this workshop?

  • Comfortable in python programming language
  • We will run the workshop code using Google Collab (free) - no other setup is needed!

Session Type Hands-on workshop

Audience LLM app developers, data scientists, data engineers

Technical Level Beginner - Intermediate

Prerequisites

  • Comfortable in python programming language
  • We will run the workshop using Google Collab (free) - no other setup is needed!

Duration 60 mins

Industry Cross industry

Speaker Bio https://sujee.dev/bio

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] Workshop: Hands-on with Data Prep Kit

Overview When building machine learning and data applications, a significant portion of your time will be dedicated to data wrangling - from content extraction and cleaning to de-duplication and filtering out problematic data. In this hands-on session we will explore Data Prep Kit - an open source toolkit, designed to streamline these essential tasks. Attendees will learn first hand how to use the Data Prep Kit to accelerate data preparation, improve overall data quality, and enhance the efficiency of building robust LLM applications.

Description Data Prep Kit is a comprehensive Python library that democratizes and accelerates data preparation by providing out-of-the-box solutions for common tasks. Engineered to scale from a single laptop to large cloud clusters, it has been successfully used to process terabytes of data for training IBM Granite Large Language Models (LLMs).

Data Prep Kit offers a robust feature set including duplicate elimination, advanced document and code handling, language detection (for both spoken and programming languages), removal of personally identifiable information (PII), as well as spam, hate speech, and malware detection.

More about Data Prep Kit : https://github.com/IBM/data-prep-kit

Join us for this hands-on session to explore how to use Data Prep Kit to accelerate data preparation, enhance data quality.

In this workshop we will do the following:

  • getting started with Data Prep Kit
  • Extract content from various documents (PDFs, DOCX, HTML)
  • Cleanup documents by removing excess markup
  • Detect and remove duplicate documents
  • Detect and remove low quality and spam documents

What do you need to participate in this workshop?

  • Comfortable in python programming language
  • We will run the workshop code using Google Collab (free) - no other setup is needed!

Session Type Hands-on workshop

Audience LLM app developers, data scientists, data engineers

Technical Level Beginner - Intermediate

Prerequisites

  • Comfortable in python programming language
  • We will run the workshop using Google Collab (free) - no other setup is needed!

Duration 60 mins

Industry Cross industry

Speaker Bio https://sujee.dev/bio

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] Workshop: Hands-on with Data Prep Kit

RAG (Retrieval-Augmented Generation) or fine-tuning a model, a significant portion of your time will be dedicated to data wrangling (cleaning, de-duping, removing markups, etc.). Data Prep Kit (https://github.com/IBM/data-prep-kit) can help you with data wrangling.

Noteworthy features of DPK include:

  • de-duping documents (exact dedupe and fuzzy dedupe),
  • handling documents and code,
  • language detection (spoken languages and programming languages),
  • malware detection, and
  • creating embeddings.

In this workshop, we will demonstrate implementing an end-to-end RAG pipeline using open source technologies:

  • Data Prep Kit for processing documents
  • Milvus as vector database
  • Granite 3 as the LLM

Session Type Hands-on workshop

Audience LLM app developers, data scientists, data engineers

Technical Level Intermediate

Prerequisites

  • A computer with a Python development environment is strongly recommended. setup instructions are here.
  • A FREE Replicate account - get one at replicate.com. Use this code to add some credit to your Replicate account! 💰

Industry Cross industry

Agenda

  • Welcome & introductions (5')
  • About the AI Alliance & how you can get involved (5')
  • Introduction to Milvus vector datastore by Stefan Webb from Zilliz (15’)
  • Workshop: “RAG pipeline with Data Prep Kit + Milvus + Granite” (60')
  • Q&A
  • Closing

About the instructor Sujee Maniyam (AI Engineer, Developer Advocate @ Node51) is an expert in Generative AI, Machine Learning, Deep Learning, Big Data, Distributed Systems, and Cloud technologies. He is passionate about developer education, fostering community engagement. Sujee has led numerous training sessions, hackathons, and workshops. He is also an author, open source contributor and frequent speaker at conferences and meetups.

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] RAG pipeline With Data Prep Kit + Milvus + Granite (Workshop)

RAG (Retrieval-Augmented Generation) or fine-tuning a model, a significant portion of your time will be dedicated to data wrangling (cleaning, de-duping, removing markups, etc.). Data Prep Kit (https://github.com/IBM/data-prep-kit) can help you with data wrangling.

Noteworthy features of DPK include:

  • de-duping documents (exact dedupe and fuzzy dedupe),
  • handling documents and code,
  • language detection (spoken languages and programming languages),
  • malware detection, and
  • creating embeddings.

In this workshop, we will demonstrate implementing an end-to-end RAG pipeline using open source technologies:

  • Data Prep Kit for processing documents
  • Milvus as vector database
  • Granite 3 as the LLM

Session Type Hands-on workshop

Audience LLM app developers, data scientists, data engineers

Technical Level Intermediate

Prerequisites

  • A computer with a Python development environment is strongly recommended. setup instructions are here.
  • A FREE Replicate account - get one at replicate.com. Use this code to add some credit to your Replicate account! 💰

Industry Cross industry

Agenda

  • Welcome & introductions (5')
  • About the AI Alliance & how you can get involved (5')
  • Introduction to Milvus vector datastore by Stefan Webb from Zilliz (15’)
  • Workshop: “RAG pipeline with Data Prep Kit + Milvus + Granite” (60')
  • Q&A
  • Closing

About the instructor Sujee Maniyam (AI Engineer, Developer Advocate @ Node51) is an expert in Generative AI, Machine Learning, Deep Learning, Big Data, Distributed Systems, and Cloud technologies. He is passionate about developer education, fostering community engagement. Sujee has led numerous training sessions, hackathons, and workshops. He is also an author, open source contributor and frequent speaker at conferences and meetups.

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] RAG pipeline With Data Prep Kit + Milvus + Granite (Workshop)

RAG (Retrieval-Augmented Generation) or fine-tuning a model, a significant portion of your time will be dedicated to data wrangling (cleaning, de-duping, removing markups, etc.). Data Prep Kit (https://github.com/IBM/data-prep-kit) can help you with data wrangling.

Noteworthy features of DPK include:

  • de-duping documents (exact dedupe and fuzzy dedupe),
  • handling documents and code,
  • language detection (spoken languages and programming languages),
  • malware detection, and
  • creating embeddings.

In this workshop, we will demonstrate implementing an end-to-end RAG pipeline using open source technologies:

  • Data Prep Kit for processing documents
  • Milvus as vector database
  • Granite 3 as the LLM

Session Type Hands-on workshop

Audience LLM app developers, data scientists, data engineers

Technical Level Intermediate

Prerequisites

  • A computer with a Python development environment is strongly recommended. setup instructions are here.
  • A FREE Replicate account - get one at replicate.com. Use this code to add some credit to your Replicate account! 💰

Industry Cross industry

Agenda

  • Welcome & introductions (5')
  • About the AI Alliance & how you can get involved (5')
  • Introduction to Milvus vector datastore by Stefan Webb from Zilliz (15’)
  • Workshop: “RAG pipeline with Data Prep Kit + Milvus + Granite” (60')
  • Q&A
  • Closing

About the instructor Sujee Maniyam (AI Engineer, Developer Advocate @ Node51) is an expert in Generative AI, Machine Learning, Deep Learning, Big Data, Distributed Systems, and Cloud technologies. He is passionate about developer education, fostering community engagement. Sujee has led numerous training sessions, hackathons, and workshops. He is also an author, open source contributor and frequent speaker at conferences and meetups.

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] RAG pipeline With Data Prep Kit + Milvus + Granite (Workshop)

RAG (Retrieval-Augmented Generation) or fine-tuning a model, a significant portion of your time will be dedicated to data wrangling (cleaning, de-duping, removing markups, etc.). Data Prep Kit (https://github.com/IBM/data-prep-kit) can help you with data wrangling.

Noteworthy features of DPK include:

  • de-duping documents (exact dedupe and fuzzy dedupe),
  • handling documents and code,
  • language detection (spoken languages and programming languages),
  • malware detection, and
  • creating embeddings.

In this workshop, we will demonstrate implementing an end-to-end RAG pipeline using open source technologies:

  • Data Prep Kit for processing documents
  • Milvus as vector database
  • Granite 3 as the LLM

Session Type Hands-on workshop

Audience LLM app developers, data scientists, data engineers

Technical Level Intermediate

Prerequisites

  • A computer with a Python development environment is strongly recommended. setup instructions are here.
  • A FREE Replicate account - get one at replicate.com. Use this code to add some credit to your Replicate account! 💰

Industry Cross industry

Agenda

  • Welcome & introductions (5')
  • About the AI Alliance & how you can get involved (5')
  • Introduction to Milvus vector datastore by Stefan Webb from Zilliz (15’)
  • Workshop: “RAG pipeline with Data Prep Kit + Milvus + Granite” (60')
  • Q&A
  • Closing

About the instructor Sujee Maniyam (AI Engineer, Developer Advocate @ Node51) is an expert in Generative AI, Machine Learning, Deep Learning, Big Data, Distributed Systems, and Cloud technologies. He is passionate about developer education, fostering community engagement. Sujee has led numerous training sessions, hackathons, and workshops. He is also an author, open source contributor and frequent speaker at conferences and meetups.

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] RAG pipeline With Data Prep Kit + Milvus + Granite (Workshop)

RAG (Retrieval-Augmented Generation) or fine-tuning a model, a significant portion of your time will be dedicated to data wrangling (cleaning, de-duping, removing markups, etc.). Data Prep Kit (https://github.com/IBM/data-prep-kit) can help you with data wrangling.

Noteworthy features of DPK include:

  • de-duping documents (exact dedupe and fuzzy dedupe),
  • handling documents and code,
  • language detection (spoken languages and programming languages),
  • malware detection, and
  • creating embeddings.

In this workshop, we will demonstrate implementing an end-to-end RAG pipeline using open source technologies:

  • Data Prep Kit for processing documents
  • Milvus as vector database
  • Granite 3 as the LLM

Session Type Hands-on workshop

Audience LLM app developers, data scientists, data engineers

Technical Level Intermediate

Prerequisites

  • A computer with a Python development environment is strongly recommended. setup instructions are here.
  • A FREE Replicate account - get one at replicate.com. Use this code to add some credit to your Replicate account! 💰

Industry Cross industry

Agenda

  • Welcome & introductions (5')
  • About the AI Alliance & how you can get involved (5')
  • Introduction to Milvus vector datastore by Stefan Webb from Zilliz (15’)
  • Workshop: “RAG pipeline with Data Prep Kit + Milvus + Granite” (60')
  • Q&A
  • Closing

About the instructor Sujee Maniyam (AI Engineer, Developer Advocate @ Node51) is an expert in Generative AI, Machine Learning, Deep Learning, Big Data, Distributed Systems, and Cloud technologies. He is passionate about developer education, fostering community engagement. Sujee has led numerous training sessions, hackathons, and workshops. He is also an author, open source contributor and frequent speaker at conferences and meetups.

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

[AI Alliance] RAG pipeline With Data Prep Kit + Milvus + Granite (Workshop)
Showing 16 results