In this talk, I’d be talking about Zarr, an open-source data format for storing chunked, compressed N-dimensional arrays. This talk presents a systematic approach to understanding and implementing Zarr by showing how it works, the need for using it, and a hands-on session at the end. Zarr is based on an open technical specification, making implementations across several languages possible. I’d mainly talk about Zarr’s Python implementation and show how it beautifully interoperates with the existing libraries in the PyData stack.

PyData Berlin are excited to bring you this open source workshop dedicated to contributing to pandas. This tutorial is 3 hours. We will have a break and continue with the same group of people.

pandas is a data wrangling platform for Python widely adopted in the scientific computing community. In this session, you will be guided on how you can make your own contributions to the project, no prior experience contributing required! Not only will this teach you new skills and boost your CV, you'll also likely get a nice adrenaline rush when your contribution is accepted!

If you don’t finish your contribution during the event, we hope you will continue to work on it after the tutorial. pandas offers regular new contributor meetings and has a slack space to provide ongoing support for new contributors. For more details, see our contributor community page: http://pandas.pydata.org/docs/dev/development/community.html .

PyData Berlin are excited to bring you this open source workshop dedicated to contributing to pandas. This tutorial is 3 hours. We will have a break and continue with the same group of people.

pandas is a data wrangling platform for Python widely adopted in the scientific computing community. In this session, you will be guided on how you can make your own contributions to the project, no prior experience contributing required! Not only will this teach you new skills and boost your CV, you'll also likely get a nice adrenaline rush when your contribution is accepted!

If you don’t finish your contribution during the event, we hope you will continue to work on it after the tutorial. pandas offers regular new contributor meetings and has a slack space to provide ongoing support for new contributors. For more details, see our contributor community page: http://pandas.pydata.org/docs/dev/development/community.html .

We talked about:

Aaisha’s background How homeschooling affects self-study Deciding on what to learn about Establishing whether a resource is good How Aaisha focuses on learning Deciding on what kind of project to build Find research materials Aaisha’s experience with the Data Talks Club ML Zoomcamp ML Zoomcamp projects Aaisha’s interest in bioinformatics Keeping motivated with deadlines Notes and time-tracking tools Drawbacks to self-studying Aaisha’s interest in machine learning Aaisha’s least favorable part of ML Zoomcamp Helping people as a way to learn Using ChatGPT as a “study group” Is it possible to use self-studying to learn high-level topics Switching topics to avoid burnout Aaisha’s resource recommendations

Links:

LinkedIn: https://www.linkedin.com/in/aaisha-muhammad/ Twitter: https://twitter.com/ZealousMushroom Github: https://github.com/AaishaMuhammad Website: http://www.aaishamuhammad.co.za/

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Shir’s background Debrief culture The responsibilities of a group manager Defining the success of a DS manager The three pillars of data science management Managing up Managing down Managing across Managing data science teams vs business teams Scrum teams, brainstorming, and sprints The most important skills and strategies for DS and ML managers Making sure proof of concepts get into production

Links:

The secret sauce of data science management: https://www.youtube.com/watch?v=tbBfVHIh-38 Lessons learned leading AI teams: https://blogs.intuit.com/2020/06/23/lessons-learned-leading-ai-teams/ How to avoid conflicts and delays in the AI development process (Part I): https://blogs.intuit.com/2020/12/08/how-to-avoid-conflicts-and-delays-in-the-ai-development-process-part-i/ How to avoid conflicts and delays in the AI development process (Part II): https://blogs.intuit.com/2021/01/06/how-to-avoid-conflicts-and-delays-in-the-ai-development-process-part-ii/ Leading AI teams deck: https://drive.google.com/drive/folders/1_CnqjugtsEbkIyOUKFHe48BeRttX0uJG Leading AI teams video: https://www.youtube.com/watch?app=desktop&v=tbBfVHIh-38

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Nadia’s background Academic research in software engineering Design patterns Software engineering for ML systems Problems that people in industry have with software engineering and ML Communication issues and setting requirements Artifact research in open source products Product vs model Nadia’s open source product dataset Failure points in machine learning projects Finding solutions to issues using Nadia’s dataset and experience The problem of siloing data scientists and other structure issues The importance of documentation and checklists Responsible AI How data scientists and software engineers can work in an Agile way

Links:

Model Card: https://arxiv.org/abs/1810.03993 Datasheets: https://arxiv.org/abs/1803.09010 Factsheets: https://arxiv.org/abs/1808.07261 Research Paper: https://www.cs.cmu.edu/~ckaestne/pdf/icse22_seai.pdf Arxiv version: https://arxiv.org/pdf/2110.

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Aleksander’s background The difficulty of selling data stack as a service How Aleksander got into consulting The Mom Test – extracting feedback from people User interviews Why Aleksander’s data stack as a service startup was not viable How Aleksander decided to switch to consulting Finding clients to consult Figuring out how to position your services Geographical limitations Figuring out your target audience The importance of networking and marketing Pricing your services The pitfalls of daily and hourly pricing and how to balance incentives Is Germany a good place to found a company? Aleksander’s book recommendations

Links:

LinkedIn: https://www.linkedin.com/in/alkrusz/ Twitter: https://twitter.com/alkrusz Website: www.leukos.io

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Ruslan’s background Fighting procrastination and perfectionism What is biohacking? The role of dopamine and other hormones in daily life How meditation can help The influence light has on our bodies Behavioral biohacking Daylight lamps and using light to wake up Sleep cycles How nutrition affects productivity Measuring productivity Examples of unsuccessful biohacking attempts Stoicism, voluntary discomfort, and self-challenges Biohacking risks and ways to prevent them Coffee and tea biohacking Using self-reflection and tracking to measure results Mindset shifting Stoicism book recommendation Work/life balance Ruslan’s biohacking resource recommendation

Links:

LinkedIn: https://www.linkedin.com/in/ruslanshchuchkin/

ree data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Parvathy’s background Brainstorming sessions with nonprofits to establish data maturity Example of an Analytics for a Better World project The overall data maturity situation of nonprofits vs private sector Solving the skill gap Publicly available content The Analytics for a Better World Academy The Academy’s target audience How researchers can work with Analytics for a Better World Improving data maturity in nonprofit organizations People, processes, and technology Typical tools that Analytics for a Better World recommends to nonprofits Profiles in nonprofits Does Analytics for a Better World has a need for data engineers? The Analytics for a Better World team Factors that help organizations become more data-driven Parvathy’s resource recommendations

Links:

LinkedIn: https://www.linkedin.com/in/parvathykrishnank/ Twitter:  https://twitter.com/ABWInstitute Github: https://github.com/Analytics-for-a-Better-World Website:  https://analyticsbetterworld.org/

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about: 

Dania’s background Founding the AI Guild Datalift Summit Coming up with meetup topics Diversity in Berlin Other types of diversity besides gender The pitfalls of lacking diversity Creating an environment where people can safely share their experiences How the AI Guild helps organizations become more diverse How the AI guild finds women in the fields of AI and data science Advice for people in underrepresented groups Organizing a welcoming environment and creating a code of conduct AI Guild’s consulting work and community AI Guild team Dania’s resource recommendations Upcoming Datalift Summit

Links:

Call for Speakers for the #datalift summit (Berlin, 14 to 16 June 2023): https://eu1.hubs.ly/H02RXvX0 Coded Bias documentary on Netflix: https://www.netflix.com/de/title/81328723#:~:text=This%20documentary%20investigates%20the%20bias,flaws%20in%20facial%20recognition%20technology. Book Weapons of Math Destruction by Cathy O'Neil: https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction Book Lean In by Sheryl Sandberg: https://en.wikipedia.org/wiki/Lean_In

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Tatiana’s background Going from academia to healthcare to the tech industry What staff engineers do Transferring skills from academia to industry and learning new ones The importance of having mentors Skipping junior and mid-level straight into the staff role Convincing employers that you can take on a lead role Seeing failure as a learning opportunity Preparing for coding interviews Preparing for behavioral and system design interviews The importance of having a network and doing mock interviews How much do staff engineers work with building pipelines, data science, ETC, MPOps, etc.? Context switching Advice for those going from academia to industry The most exciting thing about working as an AI staff engineer Tatiana’s book recommendations

Links:

LinkedIn: https://www.linkedin.com/in/tatigabru/  Twitter:  https://twitter.com/tatigabru Github: https://github.com/tatigabru Website:  http://tatigabru.com/

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Jekaterina’s background How Jekaterina started freelancing Jekaterina’s initial ways of getting freelancing clients How being a generalist helped Jekaterina’s career Connecting business and data How Jekaterina’s LinkedIn posts helped her get clients Jekaterina’s work in fundraising Cohorts and KPIs Improving communication between the data and business teams Motivating every link in the company’s chain The cons of freelancing Balancing projects and networking The importance of enjoying what you do Growing the client base In the office work vs working remotely Jekaterina’s advice who people who feel stuck Jekaterina’s resource recommendations

Links:

Jekaterina's LinkedIn: https://www.linkedin.com/in/jekaterina-kokatjuhha/

Join DataTalks.Club: https://datatalks.club/slack.html

We talked about

Chris’s background Switching careers multiple times Freedom at companies Chris’s role as an internal consultant Chris’s sabbatical ChatGPT How being a generalist helped Chris in his career The cons of being a generalist and the importance of T-shaped expertise The importance of learning things you’re interested in Tips to enjoy learning new things Recruiting generalists The job market for generalists vs for specialists Narrowing down your interests Chris’s book recommendations

Links:

Lex Fridman: science, philosophy, media, AI (especially earlier episodes): https://www.youtube.com/lexfridman Andrej Karpathy, former Senior Director of AI at Tesla, who's now focused on teaching and sharing his knowledge: https://www.youtube.com/@AndrejKarpathy Beautifully done videos on engineering of things in the real world: https://www.youtube.com/@RealEngineering Chris' website: https://szafranek.net/ Zalando Tech Radar: https://opensource.zalando.com/tech-radar/ Modal Labs, new way of deploying code to the cloud, also useful for testing ML code on GPUs: https://modal.com Excellent Twitter account to follow to learn more about prompt engineering for ChatGPT: https://twitter.com/goodside Image prompts for Midjourney: https://twitter.com/GuyP Machine Learning Workflows in Production - Krzysztof Szafanek: https://www.youtube.com/watch?v=CO4Gqd95j6k From Data Science to DataOps: https://datatalks.club/podcast/s11e03-from-data-science-to-dataops.html

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Luke’s background Luke’s podcast - AI Game Changers How Luke helps people get jobs What’s changed in the recruitment market over the last 6 months Getting ready for the interview process Stage “zero” – the filter between the candidate and the company Preparing for the introduction stage – research and communication Reviewing the fundamentals during preparation Preparing for the technical part of the interview Establishing the hiring company’s expectations Depth vs breadth Overly theoretical and mathematical questions in interviews Bombing (failing) in the middle of an interview Applying to different roles within the same company Luke’s resource recommendations

Links:

Luke's LinkedIn: https://www.linkedin.com/in/lukewhipps/

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Pauline’s background Pauline’s work as a manager at IBM What is indie hacking? Pauline initial indie hacking projects Getting ready for launch Responsibilities and challenges in indie hacking Pauline’s latest indie hacking project Going live and marketing Challenges with Unreal Me Staying motivated with indie hacking projects Skills Pauline picked up while doing indie hacking projects Balancing a day job and indie hacking Micro SaaS and AboutStartup.io How Pauline comes up with ideas for projects Going from an idea on paper to building a project Pauline’s Twitter success Connecting with Pauline online Pauline’s indie hacking inspiration Pauline’s resource recommendation

Links:

Website: https://wintopy.io/ Pauline's Twitter: https://twitter.com/Pauline_Cx Pauline's LinkedIn: https://www.linkedin.com/in/paulineclavelloux/  Blog about Indiehacking: https://aboutstartup.io

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Johanna’s background Open science course and reproducible papers Research software engineering Convincing a professor to work on software instead of papers The importance of reproducible analysis Why academia is behind on software engineering The problems with open science publishing in academia The importance of standard coding practices How Johanna got into research software engineering Effective ways of learning software engineering skills Providing data and analysis for your project Johanna’s initial experience with software engineering in a project Working with sensitive data and the nuances of publishing it How often Johanna does hackathons, open source, and freelancing Social media as a source of repos and Johanna’s favorite communities Contributing to Git repos Publishing in the open in academia vs industry Johanna’s book and resource recommendations Conclusion

Links:

The Society of Research Software Engineering,  plus regional chapters: https://society-rse.org/ The RSE Association of Australia and New Zealand: https://rse-aunz.github.io/ Research Software Engineers (RSEs) The people behind research software: https://de-rse.org/en/index.html The software sustainability institute: https://www.software.ac.uk/ The Carpentries (beginner git and programming courses): https://carpentries.org/ The Turing Way Book of  Reproducible Research: https://the-turing-way.netlify.app/welcome

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Marysia’s background What data-centric AI is Data-centric Kaggle competitions The mindset shift to data-centric AI Data-centric does not mean you should not iterate on models How to implement the data-centric approach Focusing on the data vs focusing on the model Resources to help implement the data-centric approach Data-centric AI vs standard data cleaning Making sure your data is representative Knowing when your data is good enough The importance of user feedback “Shadow Mode” deployment What to do if you have a lot of bad data or incomplete data Marysia’s role at PyData How Marysia joined PyData The difference between PyData and PyCon Finding Marysia online

Links:

Embetter & Bulk Demo: https://www.youtube.com/watch?v=L---nvDw9KU

Free data engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

How do you turn raw, unprocessed, or malformed data into dynamic, interactive web visualizations? In this practical book, author Kyran Dale shows data scientists and analysts--as well as Python and JavaScript developers--how to create the ideal toolchain for the job. By providing engaging examples and stressing hard-earned best practices, this guide teaches you how to leverage the power of best-of-breed Python and JavaScript libraries. Python provides accessible, powerful, and mature libraries for scraping, cleaning, and processing data. And while JavaScript is the best language when it comes to programming web visualizations, its data processing abilities can't compare with Python's. Together, these two languages are a perfect complement for creating a modern web-visualization toolchain. This book gets you started. You'll learn how to: Obtain data you need programmatically, using scraping tools or web APIs: Requests, Scrapy, Beautiful Soup Clean and process data using Python's heavyweight data processing libraries within the NumPy ecosystem: Jupyter notebooks with pandas+Matplotlib+Seaborn Deliver the data to a browser with static files or by using Flask, the lightweight Python server, and a RESTful API Pick up enough web development skills (HTML, CSS, JS) to get your visualized data on the web Use the data you've mined and refined to create web charts and visualizations with Plotly, D3, Leaflet, and other libraries

We talked about:

Sadat’s background Sadat’s backend engineering experience Sadat’s pivot point as a backend engineer Sadat’s exposure to ML and Data Science Sadat’s Act Before you Think approach (with safety nets) Sadat’s street cred and transition into management The hiring process as an internal candidate The importance of people management skills The Brag List The most difficult part of transitioning to management Focusing on projects and setting milestones Sadat’s transition from EM to data science management How much domain knowledge is needed for management? The main difference between engineering and management How being an EM helped Sadat transition no DS management 53:32 Transitioning to DS management from other roles How to feel accomplished as a manager Sadat’s book recommendations Sadat’s meetups

Links:

Sadat's Meetup page: https://www.meetup.com/berlin-search-technology-meetup/ Meetup event "Bias in AI: how to measure it and how to fix it event": https://www.meetup.com/data-driven-ai-berlin-meetup/events/289927565/

ML Zoomcamp: https://github.com/alexeygrigorev/mlbookcamp-code/tree/master/course-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Irina’s background Irina as a mentor Designing curriculum and program management at AI Guild Other things Irina taught at AI Guild Why Irina likes teaching Students’ reluctance to learn cloud Irina as a manager Cohort analysis in a nutshell How Irina started teaching formally Irina’s diversity project in the works How DataTalks.Club can attract more female students to the Zoomcamps How to get technical feedback at work Antipatterns and overrated/overhyped topics in data analytics Advice for young women who want to get into data science/engineering Finding Irina online Fundamentals for data analysts Suggestions for DataTalks.club collaborations Conclusions

Links:

LinkedIn Account: https://www.linkedin.com/in/irinabrudaru/

ML Zoomcamp: https://github.com/alexeygrigorev/mlbookcamp-code/tree/master/course-zoomcamp

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html