Woohoo, we’re halfway there! Having just finished my 6th week of the Data Engineering Bootcamp in Python at Northcoders, I thought it would be interesting to reflect on what we have covered already. Since the course ends with a project phase, we are now actually over halfway the ‘theory’ part of the course. However the learning will likely never stop.

I take notes during each lecture using the free note software Obsidian, and add any extra nuggets of knowledge I discover during the practicals and additional research. It’s also a great way to reflect on what I’ve learned.

Week 1: Getting Started

I would love to say the first week eased us into the course, but we started learning new things right away. We covered some general topics such as where to find good documentation and how to use Git and GitHub. (We use GitHub a lot, and you can probably tell by my profile exactly when my course started!) And we also dove straight into code, for example learning to use Regex to locate patterns in strings. 

We had all previously completed an Entry Challenge in JavaScript and prepared with a pre-course in Python, but it was still quite a hill to climb.

Week 2: Test Driven Development

In week two, we started to get into Test Driven Development (TDD). This is a core of the work ethos at Northcoders. In effect, you write a test for your code before even writing a line of your actual code. You aim for Red -> Green tests. You start off writing a test and ensure it fails (Red) for the right reasons, then work on getting it to Green in your main code block. Then build on the test for additional functionality. 

Week 3: Object Oriented Programming

In the third week, we built on our Functional Python knowledge, where you build a series of functions to make things work, and expanded into Object Oriented Programming (OOP). This involves writing Classes that can contain multiple functions that are often designed to work together. Video games often use this, and as an example we built a basic Pokemon-style battler. It was also the last week for us before a Christmas break. It was great having the tree up and a Christmas T-Rex outside my window!

Week 4: Review Day

Returning from Christmas, we had an important milestone in the course: Review Day! We learned a few new topics such as the use of Generators, File Input and Output and writing Python in the correct style (so many PEP’s to reference), then had a full day of challenges to test our knowledge and make sure we were keeping up with the course. 

Without giving too much away, for Review Day we had to solve katas based on most of the topics covered in the previous four weeks. Overall, the challenges were comparable to the easier or middle difficulty tasks we’d already done before. It's also open book, so you can use any resource at hand, including Northcoders notes and Google! As long as you pace yourself, it’s not too daunting. During the day, someone from Northcoders will pop in to check on how you’re doing, and at the end of the day you’ll (hopefully) receive a message saying you’ve done well!

Week 5: SQL

By now we have learnt the fundamentals of Python and how to tackle many problems. But as this is a Data Engineering course, it involves more than just Python. Here we start to learn about the SQL language, using the popular PostgreSQL system and PG8000 framework. SQL is how we interact with databases and the language is annoyingly simple. One moment I was hating it, then the next I was loving it, only to hate it again when trying to join tables and aggregate data…

Week 6: Servers

Once you have databases, you need to be able to interact with them. Week six builds on Postgres with servers. To start with, we worked with locally hosted servers. After writing them manually within Python for a day to learn the background, we transitioned to a system called FastAPI. If only everything was as pleasant as FastAPI, life would be so much better. The documentation is clear and easy to read, and it will also auto-document your work and give you ways to test it!

Now that we have learned about locally-hosted servers, next week we will move onto AWS, the cloud! I’ve already learned a lot over the past 6 weeks, so I’m excited to keep building on this for the rest of the bootcamp.