We talked about: 

Nicolas’ background The tech talent market in different countries Hiring data scientists vs data engineers A spike in interest for data engineering roles The importance of recruiters having  technical knowledge The main challenges of hiring data engineers The difference in hiring junior, mid, and senior level data engineers Things recruiters look for in people who switch to a data engineering role The importance of knowing cloud tools The importance of knowing infrastructure tools Preparing for the interview The importance of a formal education The importance having a project portfolio How your current domain influence the interview Conclusion

Links: 

Nicolas' Twitter: https://twitter.com/n_rassam  Nicolas' LinkedIn: https://www.linkedin.com/in/nicolasrassam/  Onfido is hiring: https://onfido.com/engineering-technology/  Interview with Alicja about recruiting data scientists: https://datatalks.club/podcast/s07e02-recruiting-data-professionals.html Webinar "Getting a Data Engineering Job" with Jeff Katz: https://eventbrite.com/e/310270877547

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Juan Pablo's Backround Data engineering resources Teaching calculus Transitioning to Analytics Data Analytics bootcamp Getting money while studying Going to meetups to get a job Looking for uncrowded doors Using LinkedIn Portfolio Talking to people on meetups Eight tips to get your first analytics job Consider contracts and temporary roles Getting experience with non-profits Create your own internship Networking Website for hosting a portfolio I’m a math teacher. What should I learn first? Analytics engineering Best suggestion: keep showing up Networking on online conferences Communication skills and being organized

Links:

Website: https://www.thatjuanpablo.com/ Twitter: https://twitter.com/thatjuanpablo BROKE teacher to FAANG engineer Twitter thread: https://twitter.com/thatjuanpablo/status/1475806246317875203 LinkedIn: https://www.linkedin.com/in/thatjuanpablo/

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Ellen’s background Why Ellen switched from data science to data engineering The overlap between data science and data engineering Skills to learn and improve for data engineering Ways to pick up and improve skills (advice for making the transition) What makes a data engineering course “good” Languages to know for data engineering The easiest part of transitioning into data engineering The hardest part of transitioning into data engineering Common data engineering team distributions People who are both data scientists and data engineers Pet projects and other ways to pick up development skills Dealing with cloud processing costs (alerts, billing reports, trial periods) Advice for getting into entry level positions Which cloud platform should data engineers learn?

Links:

Twitter: https://twitter.com/ellen_koenig LinkedIn: https://www.linkedin.com/in/ellenkoenig/

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Rahul’s background What do data engineering managers do and why do we need them? Balancing engineering and management Rahul’s transition into data engineering management The importance of updating your skill set Planning the transition to manager and other challenges Setting expectations for the team and measuring success Data reconciliation GDPR compliance Data modeling for Big Data Advice for people transitioning into data engineering management Staying on top of trends and enabling team members The qualities of a good data engineering team The qualities of a good data engineer candidate (interview advice) The difference between having knowledge and stuffing a CV with buzzwords Advice for students and fresh graduates An overview of an end-to-end data engineering process

Links:

Rahul's LinkedIn: https://www.linkedin.com/in/16rahuljain/

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We don't have a new episode this week, but we have an amazing conversation with Sejal Vaidya from August

We talked about

Sejal's background Why transitioning to ML engineering Three phases of development of a project Why data engineers should get involved in ML Technologies Tips for people who want to transition Soft skills and understanding requirements Helpful resources

Resources:

ML checklist (https://twolodzko.github.io/ml-checklist.html) Machine Learning Bookcamp (https://mlbookcamp.com/) Made with ML course (https://madewithml.com) Full-stack deep learning (https://fullstackdeeplearning.com) Newsletters: mlinproduction, huyenchip.com, jeremyjordan.me, mihaileric.com Sejal's "Production ML" twitter list (https://twitter.com/i/lists/1212819218959351809)

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Sara’s background Product designer’s responsibilities Data product manager’s responsibilities Planning with the team Design thinking and product design Data PMs vs regular PMs Skill requirements for Data PMs Going from a product designer to a data product manager Case studies Resources for learning about product management Data PM’s biggest challenge Multitasking and context switching Insights from user interviews Using new, unfamiliar tools Documentation Idea generation Do Data PMs need to know ML?

Links:

Product Management Courses: https://www.lennyrachitsky.com/course and https://www.reforge.com/mastering-product-management Product Management Reading: https://svpg.com/inspired-how-to-create-products-customers-love/ and https://steveblank.com/category/customer-development/ Data Engineering for Noobs: https://www.datacamp.com/

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Natalie’s background Airbyte What is ETL? Why ELT instead of ETL? Transformations How does ELT help analysts be more independent? Data marts and Data warehouses Ingestion DB ETL vs ELT Data lakes Data swamps Data governance Ingestion layer vs Data lake Do you need both a Data warehouse and a Data lake? Airbyte and ELT Modern data stack Reverse ETL Is drag-and-drop killing data engineering jobs? Who is responsible for managing unused data? CDC – Change Data Capture Slowly changing dimension Are there cases where ETL is preferable over ELT? Why is Airbyte open source? The case of Elasticsearch and AWS

Links:

Natalie's LinkedIn: https://www.linkedin.com/in/nataliekwong/ https://airbyte.io/blog/why-the-future-of-etl-is-not-elt-but-el

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We talked about:

Andreas’s background Why data engineering is becoming more popular Who to hire first – a data engineer or a data scientist? How can I, as a data scientist, learn to build pipelines? Don’t use too many tools What is a data pipeline and why do we need it? What is ingestion? Can just one person build a data pipeline? Approaches to building data pipelines for data scientists Processing frameworks Common setup for data pipelines — car price prediction Productionizing the model with the help of a data pipeline Scheduling Orchestration Start simple Learning DevOps to implement data pipelines How to choose the right tool Are Hadoop, Docker, Cloud necessary for a first job/internship? Is Hadoop still relevant or necessary? Data engineering academy How to pick up Cloud skills Avoid huge datasets when learning Convincing your employer to do data science How to find Andreas

Links:

LinkedIn: https://www.linkedin.com/in/andreas-kretz Data engieering cookbook: https://cookbook.learndataengineering.com/ Course: https://learndataengineering.com/

Join DataTalks.Club: https://datatalks.club/slack.html

Our events: https://datatalks.club/events.html

We covered:

Barr’s background Market gaps in data reliability Observability in engineering Data downtime Data quality problems and the five pillars of data observability Example: job failing because of a schema change Three pillars of observability (good pipelines and bad data) Observability vs monitoring Finding the root cause Who is accountable for data quality? (the RACI framework) Service level agreements Inferring the SLAs from the historical data Implementing data observability Data downtime maturity curve Monte carlo: data observability solution Open source tools Test-driven development for data Is data observability cloud agnostic? Centralizing data observability Detecting downstream and upstream data usage Getting bad data vs getting unusual data

Links:

Learn more about Monte Carlo: https://www.montecarlodata.com/ The Data Engineer's Guide to Root Cause Analysis: https://www.montecarlodata.com/the-data-engineers-guide-to-root-cause-analysis/ Why You Need to Set SLAs for Your Data Pipelines: https://www.montecarlodata.com/how-to-make-your-data-pipelines-more-reliable-with-slas/ Data Observability: The Next Frontier of Data Engineering: https://www.montecarlodata.com/data-observability-the-next-frontier-of-data-engineering/ To get in touch with Barr, ping her in the DataTalks.Club group or use [email protected]

Join DataTalks.Club: https://datatalks.club/slack.html