This is a blog post about my 5 months role at Northcoders on “The IDOL Project”. It’s also a love letter to future Northcoders, and describes what your first role might be like after graduating.

Context

I was one of a team of five, led by Northcoders Tech Lead Paul Copley, on a DevIncubator project commissioned by The IDOL.

Some caveats. This blog post is a sample of one. One opinion, one role, one project, one product, one team, one organisation. It was unique, as every new role you embark on will be too, so it’s not a scientific analysis of roles on software projects based on a large sample population.

It was also “greenfield” or “no legacy” ie. we started with no code or repos whatsoever on a totally new build, and were able to make our own technology choices, within reason. You most likely won’t be on a greenfield project, as what already exists is of far greater magnitude than what is new, but you might be.

Magnitude

To give you an idea of the size of the project, here are some numbers at completion...

Onboarding

Starting work at Northcoders was excellent. My laptop was pre-configured, digital accounts such as Gmail, Slack etc. were ready to roll. The new starter pack laid everything out: internal processes & policies, what is expected & not expected of you etc. We did our first mini-sprint on Thursday & Friday of week 1. Expect something similar. If you have a lousy first week, I would say that’s a red flag. Collect enough red flags over the first month and maybe you are not in the right team or organisation.

The Client

The IDOL are “Insurance comparison specialists and bespoke product developers”. They make the software that companies like CompareTheMarket™ and GoCompare™ use for price-comparison shopping - “here is some of my personal data, find me the best price for such-n-such type of insurance” - and earn commissions when people purchase one of the returned results.

We worked with their Senior Developer(s), Scrum Master, and Product Owner.

The Product

The product is a tool for building dynamic chatbots for retail financial products. We focused on life insurance due to its complexity i.e. many decision paths before reaching pricing quotes, terminology i.e. a lot of “industry jargon” to explain to users, and high drop-off rates i.e. users who start the comparison shopping process on a website but do not complete for all sorts of reasons. In summary, a real and challenging business case, and not done before as far as we were aware.

How did we build the product ? that’s a commercial secret, so no discussion of architecture, features, platforms I am afraid.

The Team

The team comprised a Senior Developer (a Northcoders Senior Tutor, Paul Copley in our case), and 4 Northcoders graduates. Another caveat : we were all trained in the same way, using the same tools, techniques, processes, language (JS), database (PostgreSQL) etc, so getting started was quick.

Your team most likely will not be “as homogeneous”. People are different. Different skill sets, different prior experience, different ways of thinking etc. We had our differences too in a good way. One of us had a pretty good “eye for design and UX”. One of us could write code “real-fast”. One of us was often thinking “what can go wrong”. But the thing that was most consistent across the team was development techniques (TDD, Pairing, Git etc).

Besides Javascript, I think being “consistent” was the best thing I learnt at Northcoders. It’s a set of skills that enable you to start work and function as a developer with other developers you have never met nor chosen - in the same way there are a set of core skills required to be a pilot & co-pilot on any airliner in the fleet, or surgeon & anaesthetist in any operating theatre.

Our Processes

Client Reviews : We met with The IDOL in person at several review meetings to show current progress, “live-code” some chatbots, refine ongoing project priorities, and best use of time remaining etc.

Backlog : The IDOL provided us with a set of user stories on day one. We were not involved in writing them, you might be.

Documentation : Setting up the various services and credentials for the cloud architecture was complex. We shared our documentation of the setup ( in a separate repo ) for The IDOL to replicate and validate.

Scrum : I thought it was great. We picked a user story and built the product “brick by brick”. Grafting through user stories is a great feeling and measure of progress. The daily (standup) conversations around priorities, blockers, estimates, approaches, adopt yes or no ? etc. and the sprint reviews - were all part of a great learning experience.

Estimates : We estimated story points, on a fibonacci scale, and totalled them up for each sprint. Story points in Scrum represent complexity, namely how hard do you think this is to implement ? How many points is influenced by factors such as “have you done this before” ? “can you see a way to do it - at all” ? “can we use a third-party library, and should we” ? etc. So what happened ? In some of the early sprints we delivered all of the story points and more. Then we finished a sprint where we delivered about 40 out of 60+ points. Perhaps we were over-confident. Perhaps estimating is not an exact science... In the sprint review we discussed the under-delivery, adjusted out expectations, and carried on. No drama. We always had working code and tests to continue building upon in the next sprint. One of my favourite quotes is below, although its origin is hard to pin down :

“It’s Difficult to Make Predictions, Especially About the Future”.

Version Control : We used Gitlab for the repos. Local git, checkout a branch to work on your user story, task, bug etc. Commit. Pull and merge any conflicts from master (i.e. all the code other people had committed ) before pushing back up to the master branch. On small tickets, another team member reviewed and approved commits to go into CI.

Code Reviews : On large or complex changes, we sat down as a team to do a code review for learning, familiarity, and maybe spot a few things to improve before CI.

Continuous Integration (CI) : Gitlab was also new to me and I really liked it. We setup a “pipeline” to run all unit tests (Jest), integration tests ( headless Cypress ) and coverage ( Istanbul ) before our commits could join the master branch. The gitlab UI is great, and as that commit passes all tests, woohoo, another brick is laid. When your commit fails CI, then the fun starts !

Modern Javascript

A few notes and thoughts on modern JS. As mentioned we had a clean slate, no legacy code etc - you may well not.

OOP & Classes : used a couple of times, for a tree & nodes data structure.

Functional : almost entirely. map, filter, reduce etc. Alot of manipulation of arrays of objects.

this : almost never, classes were used rarely.

React : Function components (no classes), hooks, and MST (see below).

Typing : Typescript we did not use. Some 3rd party-libraries were written in Typescript. We did use hapi/joi to define the permitted shape of JS objects flowing into PostgreSQL.

State : MobX-state-tree (MST) on the React front-end. MST is a “state container” and “very opinionated about how data should be structured and updated”. Also “each tree has a shape (type information)” providing further data integrity. For a complex front-end, you build multiple stores, with defined interfaces to update the stores. ( Redux is an alternative state store ).

Testing

“If you're working as a developer, it's highly unlikely you'll just be working on your own code. You'll often inherit code written in an unfamiliar style, provided without comments. If you're asked to add additional functionality to code like this, then you'd better hope there are tests. Because otherwise you're going to spend a lot longer working out what's going on and trying not to break things than actually making changes.”

— Sam Caine, COO Northcoders.

In the preceding table of numbers, you can see that many tests were written. Personally I enjoyed writing tests as much as the code itself. The tests became part of my ‘“flow”. Think about how you can solve the problem - write some “first thoughts” pseudocode - think about how to test it - write the tests - and then write the code. Sometimes the tests are as challenging to write as the code. If you can not figure out the tests, how do you know what to code ?

Running tests over and over is basically “free” once they are automated. And Cypress was kinda mesmerising, programmatically interacting with the product via Chrome as a user would - but at blistering speed !

More importantly, we embarked on a major refactor of a part of the project, namely adopting MST. This meant changing how state was managed in React across multiple components. It was hard at the time to imagine how we could have done this refactor without the test suite. On a smaller scale, I remember making some code changes, and thinking “that will not impact anything else” and of course, the tests said “think again”.

Craftsmanship

“It’s harder to read code than to write it.”

— Joel Spolsky”

“…the ratio of time spent reading versus writing is well over 10 to 1. We are constantly reading old code as part of the effort to write new code. Because this ratio is so high, we want the reading of code to be easy even if it makes the writing harder.”

— Robert C. Martin. Clean Code: A Handbook of Agile Software Craftsmanship.

There were only 5 of us. It could have been 50 or even 500. You will spend a lot of time reading code, other people’s code, and your own code. Ball-park ? less than 50% of my time was writing new code, the other 50% being standups, thinking before coding, code reviews, spiking etc. I was often mindful about what would happen when other developers at The IDOL or Northcoders or who knows... took the codebase on “cold” and started reading our code.

One day I realised after about half an hour, “@!&@$ I wrote this”. You will write code that you forget, totally. Why did I code it this way ? What insight do the tests give me ? What was the requirement or user story ? What happens if it errors out ? etc, and that’s just your own code.

When you read someone else’s code, it’s like trying to read their mind. The documentation, comments, descriptiveness, ease of reasoning etc. all make life easier, faster, and more productive for the next person who reads your code. So every day my thinking included : I am also writing :

“A love letter to someone I might never meet 🧡”.

Some things we did not do

• NoSQL
• Microservices
• Performance optimisation of React
• Costings for the Cloud Services
• Stress Testing i.e.how much load could the application handle
• Penetration Testing for security problems with the application
• Continuous Deployment (CD)

It is OK to feel daunted. Shout and get some help.

I had a few “wobbles”.

Wobble #1. At the end of week 2, it hit me. The code base was growing faster than I could keep up with, meaning I could not read every new line of code, nor understand everything about DialogFlow that was being learnt by the team. It felt like I was falling behind. My boss was great, “you have to let it go” he said which was very reassuring. It’s just not possible or required to stay on top of everything everyone else is writing.

Wobble #2. About 2/3rds into the project I had another wobble. A ticket to build a major front-end component. At the outset I thought my brain was going to explode : combining the user story, React, Formik, Styled Components, “state”, API endpoints to consume, tests, mocking, and more. Again my boss was excellent. He reassured me that ‘this is hard Phil”. Start small, follow the “happy path”, get something simple working and build upon it. We sat down together for an afternoon, and after that I was rather relieved and able to make progress.

Conclusion

We finished all of the user stories, and more. In my former “IT Project Management” roles it was unusual for everything to be finished. The IDOL stopped adding user stories and deferred a few others, as we were reaching the point where User Acceptance Testing ( feedback from users using the product ) was the next step. So basically we finished on time and substantially delivered what was contracted for, which felt very rewarding indeed.

The team was top notch, and my 5 months at Northocoders was great too. My boss was excellent, both technical and managerial. I hope yours is too.

Epilogue

For me personally, The IDOL project was fantastic. I did not want it to end. Fortuntately, the project we were contracted to complete was substantially completed, but sadly the end came about for commercial reasons, and coincided with the beginning of the coronavirus outbreak. In your future career, things will be stopped or cancelled for non-technical reasons, even non-IT reasons. Sometimes a business just has to focus elsewhere right now eg. to react to a competitor, implement new regulation, or respond to a pandemic.