The Fellowship of the Cloud Cloud Engineering Project
Full-stack Web Application Deployment
Final project for the Northcoders Cloud Engineering Bootcamp
The Fellowship of the Cloud Demo Video | Northcoders Project Presentations
Team Fellowship of the Cloud
Mike
Fabio
Keven
Tech Stack
We used Terraform, Docker, Dockerhub, Kubernetes, AWS (EKS, VPC, EC2, S3, CloudFront, Loadbalancer, ECR, IAM, RDS, Lambda, SES, SQS), GitHub, GitHub actions, ArgoCD, CircleCI, Java (with Springboot), Postgres, React (with Vite), Python, Prometheus, Grafana, Linux & MacOS, Bash/ZSH, Zoom, Agile/Scrum, Miro, Trello, Draw.io, VSCode, Slack.
Some of the technologies (for example the languages used) were dictated by other stakeholders, others (like terraform, circleCi, ArgoCD, GitHub) we had previous experience in. We were constrained to the use of AWS by the organisation. We mainly decided to use the technologies that we did (terraform, kubernetes, docker etc) on the basis that they are widely used in the industry, allowing us to showcase our knowledge in a way that would provide a wider range of opportunities in the future. Languages like terraform are agnostic, which allows for easy integration with new teams using technologies we haven't used before.
We faced plenty of challenges due to the scope of this project being far wider than we have experienced previously, and due to deciding to stretch ourselves by using new technologies that we have not utilised before. We had issues that broadly fit into three categories; knowledge gaps, errors in code, and permissions. Knowledge gaps: These challenges were caused by attempting things we had not learned during the course. A good example of this was using CloudFront for the first time, as we had to research how CloudFront worked, and how we could get around problems connecting the front and back ends due to http/https conflicts. Errors in code: These challenges arose when we had made a mistake in the code. They almost always coincided with knowledge gap challenges, so it was difficult working out whether we were tackling the issue in the right way and had just made an error, or were trying to solve the issue in completely the wrong way. A good example of this was when using Github actions to build and push the frontend react app to the S3 bucket. We had a small typo in the IAM policy that granted permission, and this meant that Github was not granted the right access. Permissions: Working as a team on this project threw up an entirely new challenge that we had not yet experienced during the course, permissions. We initially thought about how we could each use our own AWS accounts, but quickly realised that we would need to use the one account and create user roles for the other two members. Although we created user roles with full admin permission, we still had barriers that meant two team members couldn't use the CLI, and therefore all tereform and kubernetes commands needed to be ran from the other team members computer. While we faced plenty of challenges, we were able to overcome the vast majority of them and produce a product that we are incredibly proud of.
FAQs
You used an s3 bucket for the front end, why did you choose that option and what other options did you consider?
At the outset, an s3 bucket seemed like a simpler, cost effective option. this is because there is limited overheads in terms of how muchinfrastructure needs to be provisioned and the relative simplicity of the problem being solved - hosting a standard statis html/css/js website. Other options such as deploying a vite app on an ec2 instance would require a full vpc with subnets, building docker images and even having the app managed by kubernetes just seems like overkill. Although our s3 solution did face some issues that may have been avoided with the ec2 solution, for example routing between our front and back end might be more strightforward from a security standpoint if both apps are located on the same resource.
If you had more time, particularly in the planning and setting up phase, What would you have done differently, are there any aspects you would prioritise that you didn't get a chance to this time?
We would have spend more time on the planning phase, it would have been nice to have a diagram of how our product was intended to be architected before we provisioned any resources, we did find oursleves making decisions as we went that probably should have been made prior to the development phase. We also would have spent more time setting up our working environments, this includes things like setting up our github branches, making sure we all had access to the same remote state and access to our eks, having these responsibilities broken out in a less coherant manner did cause some friction throughout the development process. In a real world setting we might have also determined that due to the constraints of the app, such as the compute power needed, the number of users and the geographical region being served, it may have been more cost effective to host on-prem, perhaps on an inexpensive workstation.