talk-data.com

Welcome to the PyData Berlin May meetup!

We would like to welcome you all starting from 18:45. There will be food and drinks. The talks begin around 19.30.

Please provide your first and last name for the registration because this is required for the venue's entry policy. If you cannot attend, please cancel your spot so others are able to join as the space is limited.

Host: Bonial is excited to welcome you for this month's version of PyData. ************************************************************************** The Lineup for the evening

Talk 1: Beyond the Continuum: The Importance of Quantization in Deep Learning Abstract: Quantization is a process of mapping continuous values to a finite set of discrete values. It is a powerful technique that can significantly reduce the memory footprint and computational requirements of deep learning models, making them more efficient and easier to deploy on resource-constrained devices. In this talk, we will explore the different types of quantization techniques and discuss how they can be applied to deep learning models. In addition, we will cover the basics of NNCF and OpenVINO Toolkit, seeing how they collaborate to achieve outstanding performance - everything in a Jupyter Notebook, which allows you to try it at home.

Speaker: Adrian Boguszewski is an AI Software Evangelist at Intel. Adrian graduated from the Gdansk University of Technology in the field of Computer Science 7 years ago. After that, he started his career in computer vision and deep learning. As a team leader of data scientists and Android developers for the previous two years, Adrian was responsible for an application to take a professional photo (for an ID card or passport) without leaving home. He is a co-author of the LandCover.ai dataset, creator of the OpenCV Image Viewer Plugin, and a Deep Learning lecturer occasionally. His current role is to educate people about OpenVINO Toolkit. In his free time, he’s a traveler. You can also talk with him about finance, especially investments.

Talk 2: A Meat-ing of Minds Abstract: In this far wandering talk we consider the big meaty questions: What can AI learn from Neuroscience? What can Neuroscience learn from AI? Do LLMs "understand" language? What does "meaning" mean? Am I just a robot made of meat? These huge questions and more... will remain completely unanswered, but we will have a good time talking about them! AI, Neuroscience, Philosophy, Semantics. What more could you want?

Speaker: Andy Kitchen is the co-founder of Cortical Labs, a synthetic intelligence company growing live biological neurons inside computer chips and teaching them inside a matrix world. He is a 10+ year start-up veteran and turbo nerd. Beer, bad philosophy and type theory will be the death of him.

Lightning talks There will be slots for 2-3 Lightning Talks (3-5 Minutes for each). Kindly let us know if you would like to present something at the start of the meetup :)

*** NumFOCUS Code of Conduct THE SHORT VERSION Be kind to others. Do not insult or put down others. Behave professionally. Remember that harassment and sexist, racist, or exclusionary jokes are not appropriate for NumFOCUS. All communication should be appropriate for a professional audience including people of many different backgrounds. Sexual language and imagery are not appropriate. NumFOCUS is dedicated to providing a harassment-free community for everyone, regardless of gender, sexual orientation, gender identity, and expression, disability, physical appearance, body size, race, or religion. We do not tolerate harassment of community members in any form. Thank you for helping make this a welcoming, friendly community for all. If you haven't yet, please read the detailed version here: https://numfocus.org/code-of-conduct ***

Greetings, data enthusiasts!

Our May meeting is scheduled for Monday, May 13th, at 5:30 pm! We will meet in person at: Rensselaer Chamber of Commerce, 90 4th Street, Troy, NY.

Please RSVP at https://www.meetup.com/capital-area-sql-server-user-group/events/297656972/ if you are attending, so we can purchase an appropriate amount of food for everyone. Food for this month will be pizza from the venerable DeFazio's! As always, we'll order a wide assortment of toppings, ranging from cheese to some unusual stuff :-)

Our meeting schedule is as follows:

• 5:30 PM: Food, soft drinks, and networking
• 6:15 PM: Chapter news and announcements
• 6:30 PM: Presentation

We usually wrap up between 7:30 PM and 8:00 PM.

This month, Andy Levy will be joining us in-person all the way from Rochester, NY to present: Answering the Auditor's Call with Automation.

As data professionals, we are called on regularly to produce documentation for security & compliance audits. Being able to show who has what level of access to an instance is the minimum, but we're often asked for more. Collecting this information and compiling it into something usable by auditors could take you hours or even days. But with automation, you can pull it all together in a matter of minutes while you're getting that second cup of coffee from the kitchen.

Through the PowerShell demos presented in this session, you'll learn how to build documentation of your backup regimen, who has access to your databases, and show that you're staying current with SQL Server patches from Microsoft. Whether you have one SQL Server instance or one hundred, you'll be able to create a script to automatically format this data so that it's usable for your auditors - and hopefully be so complete that you don't receive follow-up questions.

Summary

Data integration and routing is a constantly evolving problem and one that is fraught with edge cases and complicated requirements. The Apache NiFi project models this problem as a collection of data flows that are created through a self-service graphical interface. This framework provides a flexible platform for building a wide variety of integrations that can be managed and scaled easily to fit your particular needs. In this episode project members Kevin Doran and Andy LoPresto discuss the ways that NiFi can be used, how to start using it in your environment, and plans for future development. They also explained how it fits in the broad landscape of data tools, the interesting and challenging aspects of the project, and how to build new extensions.

Preamble

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to run a bullet-proof data platform. Go to dataengineeringpodcast.com/linode to get a $20 credit and launch a new server in under a minute. Are you struggling to keep up with customer request and letting errors slip into production? Want to try some of the innovative ideas in this podcast but don’t have time? DataKitchen’s DataOps software allows your team to quickly iterate and deploy pipelines of code, models, and data sets while improving quality. Unlike a patchwork of manual operations, DataKitchen makes your team shine by providing an end to end DataOps solution with minimal programming that uses the tools you love. Join the DataOps movement and sign up for the newsletter at datakitchen.io/de today. After that learn more about why you should be doing DataOps by listening to the Head Chef in the Data Kitchen at dataengineeringpodcast.com/datakitchen Go to dataengineeringpodcast.com to subscribe to the show, sign up for the mailing list, read the show notes, and get in touch. Your host is Tobias Macey and today I’m interviewing Kevin Doran and Andy LoPresto about Apache NiFi

Interview

Introduction How did you get involved in the area of data management? Can you start by explaining what NiFi is? What is the motivation for building a GUI as the primary interface for the tool when the current trend is to represent everything as code? How did you get involved with the project?

Where does it sit in the broader landscape of data tools?

Does the data that is processed by NiFi flow through the servers that it is running on (á la Spark/Flink/Kafka), or does it orchestrate actions on other systems (á la Airflow/Oozie)?

How do you manage versioning and backup of data flows, as well as promoting them between environments?

One of the advertised features is tracking provenance for data flows that are managed by NiFi. How is that data collected and managed?

What types of reporting are available across this information?

What are some of the use cases or requirements that lend themselves well to being solved by NiFi?

When is NiFi the wrong choice?

What is involved in deploying and scaling a NiFi installation?

What are some of the system/network parameters that should be considered? What are the scaling limitations?

What have you found to be some of the most interesting, unexpected, and/or challenging aspects of building and maintaining the NiFi project and community? What do you have planned for the future of NiFi?

Contact Info

Kevin Doran

@kevdoran on Twitter Email

Andy LoPresto

@yolopey on Twitter Email

Parting Question

From your perspective, what is the biggest gap in the tooling or technology for data management today?

Links

NiFi HortonWorks DataFlow HortonWorks Apache Software Foundation Apple CSV XML JSON Perl Python Internet Scale Asset Management Documentum DataFlow NSA (National Security Agency) 24 (TV Show) Technology Transfer Program Agile Software Development Waterfall Spark Flink Kafka Oozie Luigi Airflow FluentD ETL (Extract, Transform, and Load) ESB (Enterprise Service Bus) MiNiFi Java C++ Provenance Kubernetes Apache Atlas Data Governance Kibana K-Nearest Neighbors DevOps DSL (Domain Specific Language) NiFi Registry Artifact Repository Nexus NiFi CLI Maven Archetype IoT Docker Backpressure NiFi Wiki TLS (Transport Layer Security) Mozilla TLS Observatory NiFi Flow Design System Data Lineage GDPR (General Data Protection Regulation)

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Support Data Engineering Podcast