Pair Programming Cloud Machine Toolkit

Tutorials
#1

Our engineering interns share what they have learned while working on the pair programming cloud machine tools.

#2

Over the past two months, I’ve dedicated considerable effort to exploring Cloud Machine Tools. During this time, I’ve learned to leverage Terraform and Ansible to set up a remote machine on Digital Ocean.

Through the use of Ansible’s local-exec module, I’ve installed Nginx on the machine. This allows users to access the machine through the HTTP port 80.

Building upon this foundation, I’ve also been able to manually add an SSL certificate to the machine using Cloudflare.

Looking ahead, my next step is to automate the SSL installation process using Ansible and Let’s Encrypt. This will help streamline the workflow and reduce the overall workload.

#3

:rocket: Reflection
As part of my engineering internship journey, I have had the privilege of delving into cloud machine tools while collaborating closely with @invisibleroads.

:wrench: Some Achievements

  • Configured IP address restrictions using Firewalld and Iptables.
  • Developed bash scripts and Ansible playbooks for seamless configuration.
  • Leveraged Terraform to swiftly provision DigitalOcean instances.
  • Automated instance setup and IP address retrieval using Terraform scripts.
  • Explored Firewalld zones and interfaces for enhanced network security.
  • Compared functionalities of Firewalld and Iptables for effective network management.

:books: What I Learned

  • Practical application of Terraform for infrastructure provisioning.
  • Working principles of Firewalld, including zones and interfaces.
  • Differences between Firewalld and Iptables.
  • Efficiency and simplicity of rule setting with Iptables.
  • Importance of automation in system configuration and security.
  • Enhanced scripting skills in bash.
  • Experience in configuring network security measures.
  • Utilizing drop instead of reject in firewalls for enhanced security.

Excited to continue learning and contributing! Moving forward, I’m eager to explore the capabilities of nftables to further enhance our network security configurations. :briefcase::sparkles:

#4

:rocket: Continuing the Journey: Building upon our previous achievements, I’m excited to share the latest developments in our exploration of cloud machine tools, while working with Salah Ahmed.

:wrench: Accomplished

  • Developed an Ansible script capable of accepting additional variables such as ssh_port, http_port, https_port, extra_ports_by_ip_address, ssh_ip_addresses, http_ip_addresses, and https_ip_addresses.
  • Crafted nftables rules from a Jinja2 template file, leveraging the Ansible template module (reference).
  • Configured nftables to start on boot using systemctl enable --now.

:books: Learnings

  • Discovered the simplicity of loading rules from a file with nftables, making it highly collaborative and conducive to peer review.
  • Expanded my knowledge of Ansible template and their simplicity. Combining Ansible with nftables allows for the creation of dynamic template configuration files.
  • Learned about Ansible’s tag and handlers functionalities, enhancing my understanding of automation processes.

Excited to continue this journey of exploration and learning! :briefcase::sparkles:

#5

Getting Started with Cloud-init: Automating Instance Initialization

Prerequisites

  • Installation of the Digital Ocean CLI tool

  • Compatible operating system such as Fedora for testing.

  • Familiarity with YAML syntax.

  • SSH public key.

Cloud-init

Cloud-init is a powerful tool that simplifies and automates the initialization process of cloud instances. In this article, we’ll explore what cloud-init is, why it’s valuable, and how to use it effectively to automate instance setup tasks.

What is Cloud-init?
Cloud-init is a multi-distribution package that handles early initialization of cloud instances. It allows you to specify configuration details, such as users, SSH keys, packages, and scripts, using a simple YAML syntax.

Why Cloud-init?
While tools like Ansible and Terraform can provision instances, cloud-init offers several advantages:

  • Early and Late Boot Stages in Instance Initialization: Creating a cloud instance involves setting up a virtual machine in the cloud. Cloud-init customizes this instance during two stages: early boot, where it configures networking, and late boot, where it handles tasks like software installation and user setup.

  • Automation: With cloud-init, you can automate instance setup entirely, eliminating manual configuration steps. Its compatibility with tools like Ansible and Puppet extends its capabilities, allowing for advanced configurations and seamless integration into existing workflows.

  • YAML-Based Configuration: Cloud-init’s YAML-based approach simplifies configuration and offers a low learning curve compared to scripting with Bash or using Terraform.

Hands-on Practice

Let’s walk through the example cloud-init configuration provided below and understand each line:

#cloud-config

package_update: true

users:
  - name: user1
    groups: interns
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - ssh-rsa <SSH_PUBLIC_KEY>

disable_root: true

write_files:
  - path: /home/user1/welcome.txt
    content: |
      Welcome to CrossCompute Cloud!
    permissions: '0644'

power_state:
  delay: 300
  mode: reboot
  message: "Rebooting the system in 5 minutes to apply updates."

packages:
  - nginx
  - python3

runcmd:
  - cat /home/user1/welcome.txt  > /usr/share/nginx/html/index.html
  - sudo systemctl enable nginx
  - sudo systemctl restart nginx
  - sudo systemctl status nginx
  • package_update: true: Updates package repositories to ensure the latest versions are available.
  • users: Configures a user named user1 with SSH access and sudo privileges.
  • disable_root: true: Disables direct root login for security purposes.
  • write_files: Creates a file named welcome.txt in the user’s home directory with a welcome message.
  • power_state: Schedules a system reboot in 5 minutes to apply updates.
  • packages: Installs Nginx and Python 3 packages. This occurs during the late boot stage, after network configuration has been completed.
  • runcmd: Executes commands to set up Nginx and display its status.

To test the script, follow these organized steps:

  1. Create Cloud-init Configuration:

    • Paste the provided script into a file named cloud-init.yaml.
    • Replace <SSH_PUBLIC_KEY> with your actual SSH public key in the ssh_authorized_keys section.

  2. Validation and Testing:

    • Install cloud-init locally for validation:

      sudo dnf install cloud-init  # On Fedora

    • Validate the YAML configuration:

      sudo cloud-init schema -c ./cloud-init.yaml

      Optionally, add the --annotate flag for more detailed validation.

  3. Execution Shell Script:

    • Create a shell script named run.sh with the following content:

      #!/bin/bash
date=$(date +%Y%m%d)
lab_name="crosscompute-lab-$date"
doctl compute droplet create --image fedora-39-x64 --size s-1vcpu-2gb --region nyc1 --ssh-keys 41326793 --user-data-file cloud-init.yaml --wait $lab_name

    • Make the script executable:

      chmod +x run.sh

  4. Digital Ocean CLI Installation:

    • Ensure you have the Digital Ocean CLI installed. Refer to the documentation for installation instructions.

  5. Running the Script:

    • Execute the script by typing ./run.sh in the terminal and wait for a few minutes for the instance creation process to complete.

    • Once the instance is created, connect to it using SSH:

      ssh user1@<ip_address>

      Replace <ip_address> with the one displayed by Digital Ocean after running the script. You should now be able to connect to the instance.

  6. Wait for Late Boot Stage:

    • Allow a few more minutes for the late boot stage to complete. This is when cloud-init installs packages and performs other configuration tasks.
    • Once the late boot stage is finished, paste the instance’s IP address into your browser to access the configured services, such as Nginx.

Conclusion

Cloud-init offers a streamlined approach to instance initialization, allowing you to automate setup tasks effectively. By leveraging its YAML-based configuration, you can simplify the process and ensure consistency across your cloud infrastructure. Start exploring cloud-init today to streamline your instance initialization workflows.

This article provides a brief overview of cloud-init and guides readers through reproducing a sample configuration. By following these steps, readers can harness the power of cloud-init to automate their instance setup tasks efficiently.

References

Cloud-init documentation

How to Automate Droplet Setup with cloud-init