TinyDevCRM Update #12: Foundational Learnings #2

This is a summary of TinyDevCRM development for the week of April 25th, 2020 to May 2nd, 2020.

Goals from last week

  • [❓] Set up a CloudFormation autoscaling definition for the compute layer
  • [❓] Set up a CloudFormation persist definition for EBS + EFS data layer

  • [❓] Set up a CloudFormation ECS definition for the container orchestration layer, with service / task / cluster definitions, and auto-pulling from ECR
  • [❓] Copy over the docker-compose setup I used for tinydevcrm-api, and load up Django + gunicorn + NGINX + PostgreSQL + static files
  • [❓] Create CloudFormation templates for ECR repositories, and get the PostgreSQL + pg_cron image pushed up to ECR as part of that effort
  • [❓] Set up CI/CD pipelines for test and production deploys with AWS CodeBuild and AWS CodePipeline

What I got done this week

  • [✔] Set up a CloudFormation autoscaling definition for the compute layer
  • [✔] Set up a CloudFormation persist definition for EBS + EFS data layer

  • [👉] Set up a CloudFormation ECS definition for the container orchestration layer, with service / task / cluster definitions, and auto-pulling from ECR
  • [✔] Copy over the docker-compose setup I used for tinydevcrm-api, and load up Django + gunicorn + NGINX + PostgreSQL + static files
  • [👉] Create CloudFormation templates for ECR repositories, and get the PostgreSQL + pg_cron image pushed up to ECR as part of that effort
  • [❌] Set up CI/CD pipelines for test and production deploys with AWS CodeBuild and AWS CodePipeline

Metrics

  • Weeks to launch (primary KPI): 3 (8 weeks after declared KPI of 1 week)
  • Users talked to total: 1

RescueTime statistics

  • 61h 27m (58% productive)
    • 21h 46m “software development
    • 10h 40m “entertainment’
    • 10h 31m “utilities”
    • 8h 0m “communication & scheduling”
    • 4h 10m “uncategorized”

iPhone screen time (assumed all unproductive)

  • Total: 37h 3m
  • Average: 5h 17m
  • Performance: 18% increase from last week

Hourly journal

https://hourly-journal.yingw787.com

Goals for next week

  • [❓] Finish creating a working CloudFormation pipeline for ECR repositories, and get the PostgreSQL + pg_cron images pushed up to ECR as part of that effort
  • [❓] Finish templating over learnings from Docker on AWS for the application side of TinyDevCRM
  • [❓] Finalize CloudFormation ECS setup for the container orchestration layer, with service / task / cluster definitions, and auto-pulling from AWS ECR
  • [❓] Ensure that rex-ray Docker volume plugin persists PostgreSQL volumes after draining EC2/ECS instances, and that the same EBS volume connected to multiple different EC2 instances keeps the same cron.schedule table.
  • [❓] Get up CI/CD pipelines for test/production deploys with AWS CodeBuild and AWS CodePipeline
  • [❓] Check back with Basecamp Personal roadmap.
  • [❓] Do a write-up of creating a SaaS product from VPC to JS, for personal understanding of creating a templated SaaS product base in the future.

Things I've learned this week

  • Non-default VPCs make for tricky debugging. I spent two days trying to debug my compute stack and why the ECS autoscaling group wasn't properly deploying and timing out instead. I should have been checking my VPC configuration instead. I had DNS resolution turned off, which meant that the Amazon Linux repository could never be resolved by the EC2 host, which meant that yum timed out, which meant that aws-cfn-bootstrap was never downloaded, which meant /opt/aws/bin/cfn-signal never existed, which meant CloudFormation never got the proper success signal.

    One configuration I found invaluable during this debugging experience is having SSH access to the underlying EC2 instances is invaluable. This meant not only setting a SSH CIDR IP with subnet mask (I used my public IP defined by my VPN), and having an EC2 key pair downloaded, but also being able to map public IPv4 addresses to the EC2 instances. This was a tricky issue, because I didn't have CloudWatch logs or CloudTrail logs configured, and they only sent barebones output. You need access to the server, and that means having SSH access.

    One thing I need to ask AWS support or figure out is running a security audit of my CloudFormation stacks (or nested stack). Ideally, production shouldn't require SSH and public IPs, but I'm guessing that professionals have staging environments for these servers and they have SSH access, then disable when moving to canry or test or prod after configuring everything properly. So baby steps. But eventually I should grow up.

    I wonder whether CloudFormation has optionally applied resources or options available. For example, if I didn't have a default SSH location, and I didn't provide one, whether it could just ignore parameter KeyName.

Subscribe to my mailing list