talk-data.com

Dear all,

we are excited to announce the October session of the AWS User Group Berlin in cooperation with Y42 and DoiT International.

We have a focus on data this time, we hope you'll enjoy the input!

Please note: you have to give your full name in the RSVP for registration at Rocket Tower security. Thanks a lot for your understanding!

========================================================================

The evening

18:30 - Warming up and networking chat

18:45 - Welcome talk

19:00 - 19:45 - Hung Dang // Git for Data: The Next DataOps Frontier

In the fast-paced world of data management, Git has been limited to code versioning. This presentation challenges that paradigm, showcasing how Git’s operational principles can redefine the Developer Experience (DX), transform data deployment, and reinforce data governance policies.

About Hung: Y42's Founder Hung Dang focussed his entire professional journey on the data analytics space, creating his first proprietary data analytics tool in 2016. This tool evolved into an independent company under the name “Mitra Intelligence”. Recognizing its immense potential, a MDAX Company (German Fortune 100) acquired “Mitra Intelligence” in 2018.

19:45 - 20:15 - Short break with snacks and drinks

20:15 - 21:00 - Rafik Saad // Patterns for effective data management in AWS data stores

Data management is an essential element of a successful cloud-based data infrastructure. This session explores common data management techniques and how they can be employed to enhance cost-efficiency and performance across various AWS data store services. Additionally, we will delve into practical implementation patterns within AWS.

About Rafik: Senior Cloud Architect at DoiT, specializing in Data and Analytics. Rafik helps startups and tech teams to enhance and modernize their cloud data platforms on AWS.

21:00 - 21:15 Closing Announcements

========================================================================

Additional Information

Would you like to host AWS UG MeetUp at your company? Register here

Would you like to speak at AWS UG MeetUp? Submit your talk here

Summary Cloud services have made highly scalable and performant data platforms economical and manageable for data teams. However, they are still challenging to work with and manage for anyone who isn’t in a technical role. Hung Dang understood the need to make data more accessible to the entire organization and created Y42 as a better user experience on top of the "modern data stack". In this episode he shares how he designed the platform to support the full spectrum of technical expertise in an organization and the interesting engineering challenges involved.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don’t forget to thank them for their continued support of this show! This episode is brought to you by Acryl Data, the company behind DataHub, the leading developer-friendly data catalog for the modern data stack. Open Source DataHub is running in production at several companies like Peloton, Optum, Udemy, Zynga and others. Acryl Data provides DataHub as an easy to consume SaaS product which has been adopted by several companies. Signup for the SaaS product at dataengineeringpodcast.com/acryl RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control. Their SDKs make event streaming from any app or website easy, and their state-of-the-art reverse ETL pipelines enable you to send enriched data to any cloud tool. Sign up free… or just get the free t-shirt for being a listener of the Data Engineering Podcast at dataengineeringpodcast.com/rudder. The most important piece of any data project is the data itself, which is why it is critical that your data source is high quality. PostHog is your all-in-one product analytics suite including product analysis, user funnels, feature flags, experimentation, and it’s open source so you can host it yourself or let them do it for you! You have full control over your data and their plugin system lets you integrate with all of your other data tools, including data warehouses and SaaS platforms. Give it a try today with their generous free tier at dataengineeringpodcast.com/posthog Your host is Tobias Macey and today I’m interviewing Hung Dang about Y42, the full-stack data platform that anyone can run

Interview

Introduction How did you get involved in the area of data management? Can you describe what Y42 is and the story behind it? How would you characterize your positioning in the data ecosystem? What are the problems that you are trying to solve?

Who are the personas that you optimize for and how does that manifest in your product design and feature priorities?

How is the Y42 platform implemented?

What are the core engineering problems that you have had to address in order to tie together the various underlying services that you integrate? How have the design and goals of the product changed or evolved since you started working on it?

What are the sharp edges and failure conditions that you have had to automate around in order to support non-technical users? What is the process for integrating Y42 with an organization’s data systems?

What is the story for onboarding from existing systems and importing workflows (e.g. Airflow d