A TerraFormed.
The past week was absolutely fantastic and filled with excitement.
We embarked on some incredible tech adventures, such as crafting our very own Ruby server and designing a custom Terraform provider in Go.
- Create A
2.0.0
Ruby Server - TerraTowns Provider Skeleton
- Terratowns Terraform Block
- TerraTowns Resource Skeleton
- TerraTowns
Home
Resource - Target TerraTowns API
- Terraform Turbocharge Workflow
- 🏠 TerraHomes to TerraTowns
- 🏙️ TerraTowns Potential Unlocked
To add to the thrill, I personally launched six TerraHomes projects to TerraTowns.
It was a whirlwind of productivity and innovation, and we can’t wait to share the fruits of our labor with all Terraformers!
Create A 2.0.0
Ruby Server
In version 2.0.0
of our project, the primary focus is on integrating the Terratown mock server into our repository.
- Sinatra In Gitpod
- Code Sinatra Server
- Bash Script for CRUDs
- Speak To Your Sinatra
- Final Server Testing
There are many ways for embedding the mock server:
A way can be;
🤔 | We can include it as a submodule |
---|---|
🐛 | Challenging to manage when issues arise |
Another way may be;
🤔 | Git checkout and refrain from committing |
---|---|
🚫 | limiting when it comes to making on-the-fly changes |
We are going with the first.
Bringing Terratown Mock Server into Our Repository
- git clone the
terratowns_mock_server
repository to our project.git clone https://github.com/ExamProCo/terratowns_mock_server.git
- cd to
terratowns_mock_server
and verify the.git
existence. - Remove the
.git
directory from the cloned repository.rm -rf .git
- Verify the contents is gone using
ls -la
. - Go a level back and verify your very critical
.git
folder for the entire project.cd .. && ls -la
Good. Take care.. 🤲
Adjust Gitpod YAML Configuration
The Gitpod in that cloned project won’t work because Gitpod operates only on the root level.
- Move the Gitpod configuration from the cloned repository to the root level of our project.
- name: sinatra init: | bundle install bundle exec ruby server.rb
Include it in the
terraform
section. - Add
cd $PROJECT_ROOT
before your terraform,aws-cli
andhttp-server
section. - Terminate the previous Gitpod workspace because It is usless.
Adjust the Repository Structure
- Change the
bin
folder toterratowns
. - Move scripts to our great
bin
directory. - Perform necessary chmod operations on these files.
- delete the come with server
bin
directory.
🎩 Understanding Sinatra
Sinatra is a web application framework. Visit sinatrarb.com for more..
The server.rb
file works with the gems listed in the Gemfile
to make the server work.
Sinatra Server Master
- Learn about Active Model in Rails as an ORM.
- Explore getter and setter methods.
- Study Rails validations and their formats.
Lets start coding the server in a single file to make it easier for you.
- Import Required Libraries:
- Import the necessary libraries
pry
andactive_model
.require 'pry' require 'active_model'
- Import the necessary libraries
- Create a Mock Database:
- Set a global variable
$home
to an empty hash. (Note: Avoid using global variables in production environments.)$home = {}
- Set a global variable
- Define the
Home
Class:- Create a Ruby class named
Home
representing a resource. - Include validations from
ActiveModel
, which provides validation functionality.class Home include ActiveModel::Validations end
- Create a Ruby class named
- Define Attributes for the
Home
Class:- Define virtual attributes using
attr_accessor
fortown
,name
,description
,domain_name
, andcontent_version
.attr_accessor :town, :name, :description, :domain_name, :content_version
- Define virtual attributes using
- Apply Validations to
Home
Class Attributes:- Use
validates
to specify validations for attributes:town
must be present and belong to a predefined list.name
must be present.description
must be present.domain_name
must have a specific format.content_version
must be an integer.validates :name, presence: true validates :description, presence: true validates :domain_name, format: { with: /\.cloudfront\.net\z/, message: "domain must be from .cloudfront.net" } validates :content_version, numericality: { only_integer: true }
Visit terratowns.cloud and explore
cooker-coke,
the permalink for validation.
- Use
- Define the
TerraTownsMockServer
Class:- Extend a class from Sinatra (
Sinatra::Base
) to create a web server.class TerraTownsMockServer < Sinatra::Base
- Extend a class from Sinatra (
- Define Error Handling Function:
- Create a method
error
to handle errors by providing an error code and message.def error code, message halt code, {'Content-Type' => 'application/json'}, {err: message}.to_json end def error_json json halt code, {'Content-Type' => 'application/json'}, json end def ensure_correct_headings unless request.env["CONTENT_TYPE"] == "application/json" error 415, "expected Content_type header to be application/json" end unless request.env["HTTP_ACCEPT"] == "application/json" error 406, "expected Accept header to be application/json" end end
- Create a method
- Define Helper Methods:
- Create helper methods
x_access_code
andx_user_uuid
to return hardcoded values. - Implement
find_user_by_bearer_token
to authenticate requests using bearer tokens.
- Create helper methods
def x_access_code
return '9b49b3fb-b8e9-483c-b703-97ba88eef8e0'
end
def x_user_uuid
return 'e328f4ab-b99f-421c-84c9-4ccea042c7d1'
end
Add the return otherwise it wont work. and I created for you find_user_by_bearer_token
method responsible for authenticating requests that use Bearer token-based authentication.
Coding Endpoints
- Create a POST Endpoint for Creating Homes:
- Define a POST route
/api/u/:user_uuid/homes
. - Ensure correct headers and user authentication.
- Parse the JSON request body.
- Validate and extract attributes from the payload.
- Create a new
Home
instance and set its attributes. - Check if validations pass; if not, return validation errors.
- Generate a UUID, mock data, and return the UUID as JSON.
- Define a POST route
post '/api/u/:user_uuid/homes' do
ensure_correct_headings()
find_user_by_bearer_token()
# puts will print to the terminal similar to a print or console.log
puts "# create - POST /api/homes"
# a begin/resurce is a try/catch, if an error occurs, result it.
begin
# Sinatra does not automatically part json bodys as params
# like rails so we need to manuall parse it.
payload = JSON.parse(request.body.read)
rescue JSON::ParserError
halt 422, "Malformed JSON"
end
# assign the payload to variables
# to make easier to work with the code
name = payload["name"]
description = payload["description"]
domain_name = payload["domain_name"]
content_version = payload["content_version"]
town = payload["town"]
# printing the variables out to console to make it eaiser
# to see or debug what we have inputed into this endpoint
puts "name #{name}"
puts "description #{description}"
puts "domain_name #{domain_name}"
puts "content_version #{content_version}"
puts "town #{town}"
# Create a new Home model and set to attributes
home = Home.new
home.town = town
home.name = name
home.description = description
home.domain_name = domain_name
home.content_version = content_version
# ensure our validation checks pass otherwise
# return the errors
unless home.valid?
# return the errors message back json
error 422, home.errors.messages.to_json
end
# generating a uuid at random.
uuid = SecureRandom.uuid
puts "uuid #{uuid}"
# will mock our data to our mock databse
# which just a global variable
$home = {
uuid: uuid,
name: name,
town: town,
description: description,
domain_name: domain_name,
content_version: content_version
}
# will jsut return uuid
return { uuid: uuid }.to_json
end
- Create a GET Endpoint for Retrieving Homes:
- Define a GET route
/api/u/:user_uuid/homes/:uuid
. - Ensure correct headers and user authentication.
- Check if the requested UUID matches the one in the mock database.
- Return the corresponding home data as JSON, or an error if not found.
- Define a GET route
get '/api/u/:user_uuid/homes/:uuid' do
ensure_correct_headings
find_user_by_bearer_token
puts "# read - GET /api/homes/:uuid"
# checks for house limit
content_type :json
# does the uuid for the home match the one in our mock database
if params[:uuid] == $home[:uuid]
return $home.to_json
else
error 404, "failed to find home with provided uuid and bearer token"
end
end
- Create a PUT Endpoint for Updating Homes:
- Define a PUT route
/api/u/:user_uuid/homes/:uuid
. - Ensure correct headers and user authentication.
- Parse the JSON request body, validate and extract attributes.
- Find the home by UUID in the mock database and update its attributes.
- Return a success message as JSON.
- Define a PUT route
# UPDATE
# very similar to create action
put '/api/u/:user_uuid/homes/:uuid' do
ensure_correct_headings
find_user_by_bearer_token
puts "# update - PUT /api/homes/:uuid"
begin
# Parse JSON payload from the request body
payload = JSON.parse(request.body.read)
rescue JSON::ParserError
halt 422, "Malformed JSON"
end
# Validate payload data
name = payload["name"]
description = payload["description"]
content_version = payload["content_version"]
unless params[:uuid] == $home[:uuid]
error 404, "failed to find home with provided uuid and bearer token"
end
home = Home.new
home.town = $home[:town]
home.domain_name = $home[:domain_name]
home.name = name
home.description = description
home.content_version = content_version
unless home.valid?
error 422, home.errors.messages.to_json
end
return { uuid: params[:uuid] }.to_json
end
- Create a DELETE Endpoint for Deleting Homes:
- Define a DELETE route
/api/u/:user_uuid/homes/:uuid
. - Ensure correct headers and user authentication.
- Find the home by UUID in the mock database and delete it.
- Return a success message as JSON.
- Define a DELETE route
# DELETE
delete '/api/u/:user_uuid/homes/:uuid' do
ensure_correct_headings
find_user_by_bearer_token
puts "# delete - DELETE /api/homes/:uuid"
content_type :json
if params[:uuid] != $home[:uuid]
error 404, "failed to find home with provided uuid and bearer token"
end
# delete from mock database
uuid = $home[:uuid]
$home = {}
{ uuid: uuid }.to_json
end
end
- Run the Server:
- Start the Sinatra server by calling
TerraTownsMockServer.run!
- Start the Sinatra server by calling
Bash Script for CRUDs
This originates from the LLM, which serves as the foundational source…
The How:
------
Write me a bash script that will send me a post request and an endpoint
------
localhost:4567/api/u/:user_uuid/homes/ with a POST it should have a headers of content type and accept application json.
------
It should expect a bearer authorization token
------
The payload json should have the follownig fileds, name description, content_version, town, domain name.
Create The Create Script
Now let’s do it.
- Before proceeding, ensure the server is stopped, as changes require a full restart. Note: Consider adding a script for automatic server restart in the future.
- Use ‘Ctrl + C’ to stop the server.
- Write a Bash script that sends a POST request to an endpoint:
- Endpoint:
localhost:4567/api/u/:user_uuid/homes/
- Headers: Content-Type and Accept should be set to “application/json.”
- Expect a Bearer Authorization Token.
- Payload JSON fields: name, description, content_version, town, domain name.
- Endpoint:
- Run
bundle exec
command to get the server back - Run the ‘terratowns/create’ script to obtain the UUID for the house.
$ ./create {"uuid":"51a588f5-18c2-461b-ba97-adfd520eb9a9"}
Perform The Read Script
It is the exact same. To run the script.
- Perform a read script using the UUID obtained from the create step.
- Run the command as follows
./read <uuid>
{ "uuid": "51a588f5-18c2-461b-ba97-adfd520eb9a9", "name": "New House", "town": "cooker-cove", "description": "A new house description", "domain_name": "3xf332sdfs.cloudfront.net", "content_version": 1 }
Perform The Update Script
To run the script;
- Perform a update script using the UUID obtained from the create step.
- Run the command as follows
./update <uuid>
When attempting an update.
It will complain. This is our code trap.
Expect an error related to domain names; this is intentional. - Investigate and correct the issue within the update action (line 216).
- Add the domain name to the update action like town
The domain names and the town should only be init once. - Run udpate again.. give same error because we have to rerun the server.
- Do new create get new uuid, read and then update.
Error. again?
The issue involves a duplicated line in the domain name, but instead of rectifying it, let’s consider appending the following information at the end.
We can employ binding.pry
to pause the program’s execution in ruby.
- Create a new record to generate a fresh
UUID
, read the data, and subsequently update it.
It is currently hanging, and this is expected behavior. - Now, let’s switch to Sinatra where it has paused.
Allowing us to interact directly with the code.
Should we refactor it?(maybe later) Lets just get the update.
Speak to Sinatra
- Type
home
=> It is now displaying. - Execute
home.domain_name
. - Inspect the payload and observe that there is no domain present. It should not be set there.
- Execute
$home[:domain_name]
—this seems to be the issue. - Realize that there is no need for
home.domain_name = domain_name
, and it’s better to keep the one with[]
. - Exit the current operation.
Relaunch the server
In sinatra review the ‘read’ operation |
---|
!Focus on the new house description |
.You can delete it now |
Server Reset and Interaction Steps
- Begin by restarting the server.
- Perform the sequence of actions: create, read, and update.
Great, everything’s in order now! Proceed to Sinatra and carefully observe all the steps.
- Append the UUID at the end, around line 240. (Note: This is not a code trap but a necessary fix.)
- Execute the ‘delete’ operation, and you’ll see it tell you about the UUID.
{ "err": "failed to find home with provided uuid and bearer token" }
- Do new create to get new uuid,
{"uuid":"82966322-962d-4910-b9e9-a4013a765730"}
- Do the read script
{ "uuid": "82966322-962d-4910-b9e9-a4013a765730", "name": "New House", "town": "cooker-cove", "description": "A new house description", "domain_name": "3xf332sdfs.cloudfront.net", "content_version": 1 }
- Do the Update;
{"uuid":"82966322-962d-4910-b9e9-a4013a765730"}
- And end it with Delete;
Our CRUDs are well set.
Note: These are all similar scripts, with slight variations such as one for POST, one for GET, and so on.
Code Considerations
- Create, read, update, and delete scripts have been provided.
- Ensure that they return the expected results.
- Has noticed another code trap that has gone unnoticed? (there is)
- Are there any thoughts on refactoring the code?
At this point our server is well set!
Once all previous steps are completed successfully, We are ready to build the custom provider.
TerraTowns Provider Skeleton
Welcome to the quickstart of our completely custom terraform provider.
Typically, it’s recommended to..
Develop a custom provider in a separate repository as an independent project.
We’ll nest it, empowered, within our project for this instance.
- Create Your Custom Provider
- Provider Functionality
- Imports In Go Lanugage
- Setup Terrtowns Plugins
- Complie Your Custom Provider
mod.go
Requirement File- Build Custom Provider Skeleton
When you visit any provider on the Terraform Registry, you’ll notice a consistent naming pattern.
🖊️ | terraform-provider-name |
---|---|
🌐 | Browse and see for yourself. |
I filtered it for you on communtiy tier already.
Also..
- Consult back the custom providers stuctures and code blocks.
- Hasicups is a very good project for learning Terraform.
- Community often includes individuals who start and don’t complete their providers.
But that’s not us. we are doing it till the very end.
Over the past weeks, we’ve been actively experimenting with a variety of exciting technologies and the fun just started.
Create Your Custom Provider
We’ve successfully developed our Ruby server with the assistance of Sinatra, and now, we’re diving into Go to enhance our skills as we work on creating the custom provider.
- Create a new folder and name it
terraform-provider-terratowns
at root level. - Inside this folder, create a file named
main.go
. (Single file for simplicity) - Consolidate the code into a single file for improved readability.
- In the
main.go
file, structure the code as follows:package main func main() { // Your code here }
We always have the package main. Use (done) GPT to generate a “Hello World” program using the Go package and print the result.
- Now, let’s create a simple “Hello World” program in Go.
package main
import "fmt"
func main() {
fmt.Println("Hello, World!")
}
- Unlike languages like Ruby, Go files are compiled into binaries
- Unlike languages like Ruby, Go files are not dynamically executed.
- You compile the script, and it runs as a binary.
Go is built-in on Gitpod.
Locally I installed it previously and it took time for vars. |
---|
Check it out, A blog I posted on hashnode on feb of this 2023 |
- Run your first Go program.
(51-terratowns-skeleton)
$ go run main.go
$ go run main.go
Hello, World!
Provider Functionality
Our custom Terraform provider will provide CRUD operations for a resource.
Make a Request to LLM in the meanwhile.
Generate code for a custom Terraform provider that includes API actions designed to interact with Terraform resources. Can you confirm, so I provide you with requirements.
---
Requirements:
We aim to perform the following actions;
1. GET /api/u/:user_uuid/home
2. POST /api/u/:user_uuid/home
3. PUT /api/u/:user_uuid/home
4. DELETE /api/u/:user_uuid/home
Importing in Go
To import multiple packages in Go, you can use the following syntax:
import (
"package1"
"package2"
// Add more packages here
)
Plugin Setup
We’ll need to set up a plugin server for our provider exactly why we coded the ruby server.
- In the
main
function add our custom provider.plugin.Serve(&plugin.ServeOpts{ ProviderFunc: Provider, })
- Below the
main
function, specify the provider as a function.func Provider() *schema.Provider {}
- A function in go is just func
- Go doesn’t use classes; instead, it relies on interfaces.
Get Help From HashiCorp
We used the developer.hashicorp.com
Terraform provider setup tutorial for this.
- Retrieve the link from the
main.go
file and incorporate it into our schema. - Code the function and add the following;
var p *schema.Provider p = &schema.Provider{
- Create the provider schema
p = &schema.Provider{}
- Define the resource and data sources map inside p;
ResourcesMap: map[string]*schema.Resource{ }, DataSourcesMap: map[string]*schema.Resource{ },
- Include the necessary elements for the schema incl
endpoint
,user_uuid
andtoken
.Schema: map[string]*schema.Schema{ "endpoint": { Type: schema.TypeString, Required: true, Description: "The endpoint for hte external service", }, "token": { Type: schema.TypeString, Sensitive: true, // make the token as sensitive to hide it the logs Required: true, Description: "Bearer token for authorization", }, "user_uuid": { Type: schema.TypeString, Required: true, Description: "UUID for configuration", //ValidateFunc: validateUUID, }, }, }
- Code validateFunc like a pro by adding it like this;
validateFunc:= validateUUID
- Request GPT to generate the code for it.
Can you fill the validate UUID function for me?
Nah.
Okay np.
Google UUID Lib
-
Google provides its own library at github.com/google/uuid, which offer a solution for validation.
- Use this stackoverflow refernce for more about validation.
- Get the prompt from GPT and lets tweak it baby.
func validateUUID(v interface{}, k string) (ws []string, errors []error) { log.Print('validateUUID:start') value := v.(string) if _,err = uuid.Parse(value); err != nil { errors = append(error, fmt.Errorf("invalid UUID format")) } log.Print('validateUUID:end') }
Great and cool. Now we have to look how to make this run.
Complie Your Custom Provider
To compile a custom provider, you need a terraformrc
file.
Terraform maintains a hidden directory called .terraform.d
with a plugins
folder to store plugin binaries.
Ensure that you have a local provider directory where you place the binary files generated.
- Run the following command to build the custom provider binary:
go build -o terraform-provider-terratowns-v1.0.0
- Test the custom provider using the
.terraformrc
file. - Create a
terraformrc
file at the root of your project.
Ensure that the file name does not contain a period as it should be placed within your custom provider code. (We will use cp command.)
Inside the terraformrc
file, add the following configuration for provider installation:
provider_installation {
filesystem_mirror {
path = "/home/gitpod/.terraform.d/plugins"
include ["local.providers/*/*"]
}
direct {
exclude = ["local.providers/*/*"]
}
}
Also..building the binary can be challenging (was really hard..)
🙏 Thanks to Jason for this article.
Scripting The Build Process
A script-based approach is recommended:
- Navigate to the
bin
directory. - Create a file named
build_provider
and add the necessary content to build the binary.rm -rf ~/.terraform.d/plugins rm -rf $PROJECT_ROOT/.terraform rm -rf $PROJECT_ROOT/.terraform.lock.hcl
We started by removing previous dependencies in case.
- Create both compute env, once for each chipset on the target compute
mkrdir -p ~/.terraform.d/plugins/local.providers/local/terratowns/1.0.0//x86_64/ mkrdir -p ~/.terraform.d/plugins/local.providers/local/terratowns/1.0.0//linux_amd64/
This ensures compatibility with what yours may be using. Now make use of environment variables in your script to simplify the build process.
- Apply the environment variables for your plugin path.
PLUGIN_DIR="~/.terraform.d/plugins/local.providers/local/terratowns/1.0.0/" mkrdir -p $PLUGIN_DIR/x86_64/ mkrdir -p $PLUGIN_DIR/linux_amd64/
- Apply the environment variables for your plugin name and copy it.
PLUGIN_NAME="terraform-provider-terratowns_v1.0.0" cp $PLUGIN_NAME $PLUGIN_DIR/x86_64 cp $PLUGIN_NAME $PLUGIN_DIR/linux_amd64
First, Verify the script from here right away.
Second. We can’t build this yet
To really build the provider we need something else.
moddotgo Is A Must
You need to ensure you have a go.mod
file.
- Create a file named go.mod and add the necessary content.
- Add repo URL as The module and the version.
module github.com/ExamProCo/terraform-provider-terratowns go 1.20
- Important to Map the repo URL to the path of your workspace.
replace github.com/ExamProCo/terraform-provider-terratowns => /workspace/terraform-beginner-bootcamp-2023/terraform-provider-terratowns
- Add an open require for what we will need.
require ()
Now we are ready for the build and further troubleshoot.
Real Build Check
- Run the build script. We called it
build_provider
You will get an error about a missing import path. |
---|
Correct it by ensuring proper use of parentheses in your import statements. |
It is
()
and not{}
- Build again
You are now being asked to get the github repos required.
- Run go get to fetch any required GitHub repositories as indicated by the error message.
go get github-url-lib
This installs the latest version of the module.
- Building again result in a new
go.sum
file.
The go.mod file now includes more URL paths, which you didn’t have to manually write.
- Building again.. also know that the first time may be slow due to dependency downloads.
You might encounter errors during this build process number too much..
The provider schema declaration is to replace .Resources{
with .Schema{.
Also add the comma..
This is not ruby.
Continue building and address any unused log errors.
- Remove or uncomment the log from the import in your
main.go
// "log"
Finally, after a successful build.
You will find the terraform-provider-terratowns-v1.0.0
binary.
This is so wrong. You’ll find out next release. We had traps.
It can be quite large, so consider the following;
- Observe the
go.sum
file and all the dependencies here. - Add your binairy to your
.gitignore
.# Ignore Terratowns Custom Provider Binairies All versions terraform-provider-terratowns/terraform-provider-terratowns_v*
This file now won’t be comitted and will be avoided by your version control!
Considerations
- The subsequent builds but the first should be faster.
- The binary file for your provider can be large (e.g., 20MB).
- Use
.gitignore
file to exclude it from version control.
Creating a custom Terraform provider can be a complex process
This starts the process of generating the actual binary which is also the Skeleton For our Terratowns!
If it is building for you, You are in a good shape for 2.2.0
!
Terratowns Terraform Block
Welcome to 2.2.0
our second part for working with the creation of our custom provider.
- Connecting Dots And Traps
- Configuring the Custom Provider Block
- Rebuilding the Provider After Your Code
- Idea On The Way
- Back To Plan
To get the most of this, I highly encourage you refer to the issues we dealt with for Terratowns starting with this.
Connecting Dots And Traps
We have our plugin ready from previous tag.
Let’s rebuild the binary.
./bin/build_provider
There is build errors. An issue occurred while attempting to set terraformrc
.
- There was a mistakenly executed
mkdir
command. - add an
=
betweeninclude
and["local.providers/*/*"]
in terraformrc`.
Double execute the script and it should now work fine, this generates;
- The binary which is what we want.
- Another folder that starts with a tilde
(~)
, terminate it.
If you back to my previous branch, you should notice that I got that and left it for reference.
- In the script replace
(~)
with/home/gitpod
. - Correct
PROJET_ROOT
withPROJECT_ROOT
. - Change the second
(~)
inrm -rf
to/home/gitpod
.
|Please correct the script by updating the path for the ‘plugin_dir’ variable| |—|
These corrections eliminated errors, resulting in the successful generation of the binary.
- To confirm the binary was generated
Check it interraform-provider-terratowns
.
Now that our library is configured, it’s time to integrate it with our Terraform setup. This corresponds to version 2.2.0
of our project.
Configuring the Custom Provider Block
The next step is to configure the custom provider within Terraform:
- Update
main.tf
file to specify the Terraform block with our provider.required_providers { terratowns = { source = "local.providers/local/terratowns" version = "1.0.0" } }
- Below the Terraform block, create the provider block itself.
provider "terratowns" { what is next goes here }
- Specify the required the endpoint, service UUID, and token.
endpoint = "http://localhost:4567" user_uuid="get-it-from-teacherseat-profile" token="get-it-from-teacherseat-settings"
The required UUID and token values were obtained from a mock created by the script.
Later this will both come from the ExamPro Platform. (It is already there)
Rebuilding the Provider After Your Code
-
Build our provider again to get an updated binary.
-
Initialize Terraform, and execute
terraform init
.
❌ ERROR |
---|
An error occurred during initialization due to a missing include. |
- The build script was executed again, followed by
terraform init
.
This now resolves our issue.
Idea On The Way
When you perform actions like that, having different log levels in place can greatly simplify the process.
Effective logging can make troubleshooting easier. In this release, log levels were configured for debugging purposes:
- Add the log level using the command
TF_LOG=DEBUG tf init
- Enable Debug mode was in the environment configuration within the Terraform block in GitPod file.
tasks: - name: terraform env: TF_LOG: DEBUG
We can now debug and get good stuff.
When things go awry in our Go code, these logs will prove invaluable for troubleshooting. This verbose mode is great idea.
Back To Plan
We stopped at the init, where we resolved our issue.
- Run
tf plan
to quickstart our code.
To ensure that the changes made were effective, the following steps were taken:
- Make sure module blocks were commented to avoid interference.
- Make sure all output lines in the root also.
tf plan will now work with No changes. . |
---|
Your infrastructure matches the configuration |
- Run
terraform apply
or justtfaa
again to push our code with auto approve using our previous configured alias.
Things are working great now!
Applying the configuration will produce an empty Terraform state |
---|
This is actually a positive and desired outcome |
Good Stuff!
Version 2.2.0
of the TerraTown Provider saw improvements in the build script, custom provider configuration, debugging capabilities, and error resolution.
We’re also introducing the Terraform block for our new custom provider, ensuring its functionality.
2.3.0
is our final step and will involve a more in-depth exploration of Go and the coding of the actual resources for our provider.
TerraTowns Resource Skeleton
In the previous setup 2.2.0
, we did not define any resources for our provider but layed the required to do so!
Now, let’s begin with the creation of a 2.3.0
Resource Skeleton.
We’ll create a basic structure for a new resource and introduce some Go programming concepts along the way.
- Preparing and Energizing
- Writing Provider Configuration
- Setting Up a Resource
- Production Considerations
The next 2.4.0
is where we will actually go and fill those cruds as per our requirements in go.
Preparing and Energizing
Take note of the extensions you’ll need, such as Go and Golang tools.
- Before all, uncomment the validator in your
main.go
code if you didn’t remove it.
I did cause I want my code neat.
Take a look.
I coded it again.
- Add
ValidateFunc: validateUUID
, to your schema uuid block. - Also re-enable logging.
import ("log")
- Also, ensure that you are working within the Terraform tab,
We have set up debugging for this context.
Issue Processing master
- build our provider to see, you may encounter errors.
- Replace single quotes (
''
) with double quotes (""
) for the func validator. - Build again; if you encounter errors, include ‘google’ in the import statement for ‘uuid’ lib.
github.com/google/uuid v1.3.0
- Use
go get
to fetch dependencies from the specified GitHub URL
/terraform-provider-terratowns (55-terratowns-cruds) $ go get github.com/google/uuid go: upgraded github.com/google/uuid v1.3.0 => v1.3.1
Make sure you are inside the correct directory.
- Return to the our directory and build again or stay..
You may encounter more errors;
- If related to
append
, adjust the error handling toerrors
- If related to missing returns, make sure to add them as needed.
In some cases, returning
true
may be necessary, butreturn
alone can work effectively.
Keep in mind that building providers in Go can be complex, and it’s okay if you’re just starting to get the hang of it.
- If related to
- Go build again, and should work fine now.
- Proceed with
terterraform init
. You should see logs indicating a successful initialization.
- run
tf plan
and observe- No errors found!
- No infrastructure changes!
2023-10-05T21:52:23.756Z [INFO] backend/local: plan operation completed
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration and found no
differences, so no changes are needed.
This is expected at this stage aka back to black!
Writing Provider Configuration
Create a providerConfigure
function.
We will start by writing some initial code.
func providerConfigure(p *schema.Provider) schema.ConfigureContextFunc {
}
- Add the return func nested within;
return func(ctx context.Context, d *schema.ResourceData) (interface{}, diag.Diagnostics ) { }
- Add the Config for
endpoint
,token
anduuid
;config := Config{ Endpoint: d.Get("endpoint").(string), Token: d.Get("token").(string), UserUuid: d.Get("user_uuid").(string), } return &config, nil
- Fix two prints to help in the debug;
// Before Config log.Print("providerConfigure:start") // Before Return log.Print("providerConfigure:end")
- Add or uncomment if the following line before returnig
p
in provider schema func;p.ConfigureContextFunc = providerConfigure(p)
- Rebuild the provider after adding the
providerConfigure
function. you may encounter errors related to undefined elements. - To fix that, Import More Dependencies
- Import
diag
,context
packages.import ( "context" "github.com/hashicorp/terraform-plugin-sdk/v2/diag")
- Import
- For config, Define your own structure for the configuration after the
import
type Config struct { Endpoint string Token string UserUuid string }
We are all set, lets setup our resource.
Setting Up a Resource
Next, we’ll set up a resource for the provider. We will define our Cruds and code these actions as separate functions.
While we could create separate files for this, we’ll keep everything together for readability your ease of learn.
- Modify Provider Schema
- In the provider schema (provider.pro), add the resource name
terrtowns_home
.ResourcesMap: map[string]*schema.Resource{ // added the following: "terratowns_home": Resource(), },
- In the provider schema (provider.pro), add the resource name
- Resource Functions
Define the four basic CRUD actions for your resource block;
func Resource() *schema.Resource {
log.Print("Resource:start")
resource := &schema.Resource{
CreateContext: resourceHouseCreate,
ReadContext: resourceHouseRead,
UpdateContext: resourceHouseUpdate,
DeleteContext: resourceHouseDelete,
}
log.Print("Resource:start")
return resource
}
These actions are standard for every resource in a Terraform provider. We have to code a func skeleton for each.
Starting with the create;
func resourceHouseCreate(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
var diags diag.Diagnostics
return diags
}
Proceed to the read;
func resourceHouseRead(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
var diags diag.Diagnostics
return diags
}
Advance to the update;
func resourceHouseUpdate(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
var diags diag.Diagnostics
return diags
}
Wrap the skeleton with delete;
func resourceHouseDelete(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
var diags diag.Diagnostics
return diags
}
- Build Again
- Rebuild the provider after defining these functions.
./bin/build_provider
- Rebuild the provider after defining these functions.
- Fix Errors
Address any issue encountered during this build process;
- Ensure that interface definitions are in
{}
and not()
. and - Ensure there is no missing as missing commas.
-
Ensure you return the resource in the main function for all actions.
- Build Again
- Rebuild the provider one more time to ensure everything is working as expected.
$ ./bin/build_provider
After these steps, you should have empty functions in place. This marks the completion of the skeleton setup.
These are now coded, built again and pushed to the branch.
Everything later will come into a place in week-2-perfecting.
Production Considerations
In a real project, you wouldn’t commit and create pull requests for these changes;
They would typically remain in a feature branch until the entire job is finished.
In the next phase and last, we’ll start filling in the resource and making the API calls.
- OpenAI plugins are great stuff
TerraTowns Home Resource
We welcome you in this last and intense 4/4
part of our custom terraform provider creation.
- Code the Terraform Resource
- Building and Initializing
- Code Implementation
- Payload Processing in API Requests
- Testing and State
We will start by defining the schema for our resource.
I want to call it home resource.
But since its the only resource..
We call it just resource.
Remember it is still our home 😊
Compiling the Code
To check if everything compiles correctly first, run the following command:
bin/build_provider
Code the Terraform Resource
- Go to the
main.tf
file at the root level. - Add a new Terraform resource named
terrtown_home
and name ithome
.resource "terratown_home" "home" {}
- Start with Including the
name
attributes.name = "Secrets are at the core of successful businesses"
Where I read that quote?
- choose where you want to put your page e.g.;
gamers-groto
fortown
.town = "gamers-grotto"
- Use the
<< >>
syntax to configure the description, similar to EOF.description = <<DESCRIPTION Something so great and innovative. Something Amazing. As great as you. DESCRIPTION
- Retrieve the domain name from your other project (CloudFront URL)
- set the domain_name as an output.
domain_name = "very-random.cloudfront.net"
- Specify the
content_version
starting with one;content_version = 1
Building and Initializing
Run the following commands to prepare your Terraform environment:
- Run
terraform init
ERROR: failed to query available provider packages
- review your Terraform configuration files include terraformrc.
Everything looks fine..
We didnt finish anything with the code..
We still have empty resources in main.go
it may be why..
Lets keep coding and see.
Code Implementation
Next, we need to code the CRUD (Create, Read, Update, Delete) operations for our resource.
Create Action
- Implement the HTTP request and endpoint for creating a resource.
req, err := http.NewRequest("POST", config.Endpoint+"/u/"+config.UserUuid+"/homes", bytes.NewBuffer(payloadBytes)) if err != nil { return diag.FromErr(err) }
- Add authorization and your bearer token to the create request.
req.Header.Set("Authorization", "Bearer "+config.Token)
- Set headers for content type and accept.
req.Header.Set("Content-Type", "application/json") req.Header.Set("Accept", "application/json")
- Ensure that you handle any errors and parse the response.
if err != nil {
return diag.FromErr(err)
}
defer resp.Body.Close()
// parse response JSON
var responseData map[string]interface{}
if err := json.NewDecoder(resp.Body).Decode(&responseData); err != nil {
return diag.FromErr(err)
}
// StatusOK = 200 HTTP Response Code
if resp.StatusCode != http.StatusOK {
return diag.FromErr(fmt.Errorf("failed to create home resource, status_code: %d, status: %s, body %s", resp.StatusCode, resp.Status, bytes.NewBuffer(responseData)))
}
- Return the
homeUUID
from the response and set it usingd.Set
.
homeUUID := responseData["uuid"].(string)
d.SetId(homeUUID)
- Add print at the start for debugging purposes
log.Print("resourceHouseCreate:start")
Read Action
- Implement the HTTP GET request for reading a resource.
req, err := http.NewRequest("GET", config.Endpoint+"/u/"+config.UserUuid+"/homes/"+homeUUID, nil)
if err != nil {
return diag.FromErr(err)
}
- Pass the
homeUUID
in the URL. - Set the headers for the read func
req.Header.Set("Authorization", "Bearer "+config.Token) req.Header.Set("Content-Type", "application/json") req.Header.Set("Accept", "application/json")
- Parse the response data and return it.
client := http.Client{}
resp, err := client.Do(req)
if err != nil {
return diag.FromErr(err)
}
Consider the code to handle errors gracefully.
(Coming below)
Update Action
- Implement the HTTP request for updating a resource.
req, err := http.NewRequest("PUT", config.Endpoint+"/u/"+config.UserUuid+"/homes/"+homeUUID, bytes.NewBuffer(payloadBytes))
if err != nil {
return diag.FromErr(error)
}
- Set the header for the update
req.Header.Set("Authorization", "Bearer "+config.Token)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
- Use the payload from the create action but exclude domain and town.
- Set the payload data in
d.Set
for name, description, and content version.
d.Set("name",payload["name"])
d.Set("name",payload["description"])
d.Set("content_version",payload["content_version"])
Delete Action
- Implement the HTTP request for deleting a resource.
req, err := http.NewRequest("DELETE", config.Endpoint+"/u/"+config.UserUuid+"/homes/"+homeUUID, nil)
if err != nil {
return diag.FromErr(err)
}
- Set the ID if the operation is successful and a print.
d.SetId("")
log.Print("resourceHouseDelete:end")
..This repetition could potentially be refactored |
---|
try? |
Next, we need to conduct the client call following the header, both for the delete operation and similarly for the update, read, and create operations.
Payload Processing and Error Handling in API Requests
The message body should contain all the details, here is a design for your reference.
- Pay Payload for a Post
- To begin, we need to pay a payload for a post in createfunc.
payloadBytes, err := json.Marshal(payload) if err != nil { return diag.FromErr(err) }
- To begin, we need to pay a payload for a post in createfunc.
- Create Configuration
- Now, let’s return to the creation process.
- After configuring, we need to pass a payload block.
payload := map[string]interface{} { }
- Payload Formatting
- It’s important to format the payload properly.
- Consider using a JSON function to convert it into bytes and add it as an argument to the next step.
"name": d.Get("name").(string), "description": d.Get("description").(string), "domain_name": d.Get("domain_namae").(string), "town": d.Get("town").(string), "content_version": d.Get("content_version").(int64),
- Handling the Response
- After completing the payload and putting it in the body, we should receive a response.
- We’ll need to perform actions based on this response.
- Parse the response data.
var responseData map[string]interface{} if err := json.NewDecoder(resp.Body).Decode(&responseData); err != nil { return diag.FromErr(err) }
- Closing the Response Body
- Don’t forget to close the response body after you’re done with it.
- Use
resp.Body.Close()
to accomplish this.defer resp.Body.Close()
- Error Handling in Create
- Obtain the code that will detect and handle errors from the server during the create process.
- Check if
resp.StatusCode
is not equal tohttp.StatusOK
to determine if an error occurred.if resp.StatusCode != http.StatusOK { return diag.FromErr(fmt.Errorf("failed to create home resource, status_code: %d, status: %s, body %s", resp.StatusCode, resp.Status, bytes.NewBuffer(responseData))) }
- Handling Errors in Delete
- Similarly, for the delete operation, you’ll need to implement error handling.
- Check the response status code for errors.
if resp.StatusCode != http.StatusOK { return diag.FromErr(fmt.Errorf("failed to delete home resource, status_code: %d, status: %s, body %s", resp.StatusCode, resp.Status, responseData)) }
- Setting the ID Value
- In Terraform, when working with resources, it’s essential to return an ID.
- In the create step, add the following:
homeUUID := response uuid string
- Set
d
to this value and print for debug.
d.SetId("")
log.Print("resourceHouseDelete:end")
Setting the ID Value
In Terraform, it’s important to return an ID whenever you have a resource.
- In the create step, add the following:
homeUUID := response uuid string
- Set
d
to this value, e.g.,d.set(homeUUID)
.
Reading Resource Data
Now, let’s focus on the read operation.
- We need the home ID for this.
config := m.(*Config) homeUUID := d.Id()
- Revise the response status code to use an “if-else” structure.
- If it’s OK, return the data.
if resp.StatusCode == http.StatusOK {
// parse response JSON
if err := json.NewDecoder(resp.Body).Decode(&responseData); err != nil {
return diag.FromErr(err)
}
// d.set
} else if resp.StatusCode != http.StatusNotFound {
d.SetId("")
}
- Use
d.Set
for all the content to be read, such asname
,content
,desc
,domain
, etc.d.Set("name",responseData["name"].(string)) d.Set("description",responseData["description"].(string)) d.Set("domain_name",responseData["domain_name"].(string)) d.Set("content_version",responseData["content_version"].(int64))
- Consider handling cases where the status is not available to avoid config drift.
else if resp.StatusCode != http.StatusOK { return diag.FromErr(fmt.Errorf("failed to read home resource, status_code: %d, status: %s, body %s", resp.StatusCode, resp.Status, responseData)) }
Good work!
Updating Resources
For the update operation, you’ll need to use the payload from the create operation.
- Exclude the
domain
andtown
since these should not change. - Return the payload values for
name
,desc
,description
, andcontent_ver
. - Use
d.Set
to update these values.
Testing
After implementing these actions, you should test your provider.
- Build and initialize Terraform.
./bin/build_provider
We have some issues and syntax error.
- Replace
err != {
witherr != nil {
in your code to resolve the issue.
// Before
if err != {
return diag.FromErr(err)
}
// After
if err != nil {
return diag.FromErr(err)
}
- Build again baby.
./bin/build_provider
Error. HTTP error
- import the necessary package, you can add the statement as follows:
import ("net/http")
- Build again one more.
./bin/build_provider
Error. We have an
undefined.FrontErr
. - Change this with correct the usage of the identifier with this;
FromErr
- Building building!
./bin/build_provider
Another cute error, the “bytes” library is required.
- You can add the import statement like this to resolve the issue in your main.go:
import ( "bytes")
We have gone through multiple builds, and to avoid spamming you with more build errors, I will now list the errors we encountered and how we resolved them.
The Next ERROR;
- The variable
payloadBytes
is declared but not utilized on line 255 because we need to pass it instead ofnil
. - We should use
bytes.NewBuffer
to create the buffer since it’s not reading. The same adjustment should be made for thecreateAction
andreadOnly
functions.
The Next ERROR +1;
- We don’t need
responseData
for the update operation. We are not interested in it. - Please remove
responseData
and only capture the response body. - Take the
responseData
from the delete function as well.
- Building the great build!
./bin/build_provider
We’ve successfully built it now!
🤯 | It’s okay if it seems confusing at the moment; |
---|---|
💯 | Things will get easier |
🧐 | When we start examining the tfstate file |
💪 | Don’t give up King or Queen. |
Test The Magic
Does this actually now provision anything? Is it working as expected?
This is communicating with our Sinatra server.
- do
tf init
and let’s see.
We encountered a failure while querying available provider packages.
- Please ensure consistency in naming.
- Use the resource name
terratowns_home
instead ofterratown_home
.
- check your terraform providers locally.
$ terraform providers
Providers required by configuration:
.
├── provider[local.providers/local/terratowns] 1.0.0
└── provider[registry.terraform.io/hashicorp/terratown]
- do it again after the update! Perfect.
Providers required by configuration:
.
└── provider[local.providers/local/terratowns] 1.0.0
This change is necessary because in the provider schema function of main.go
, it is defined as teratown_home
.
- Double check
tf init
now. Works! terraform plan
and let’s see.
We’ve got the resource! A custom resource has been planned
!
- The mock server is not a real server,
- This won’t behave exactly the same way as the actual server.
We can play a little bit around before targetting the terratown.
- Apply your changes with
terraform apply
for too much errors to come ofc.
Step | Error Description | Resolution |
---|---|---|
1 | Spelling mistake: domain_namae -> name |
Update the variable name to “name” |
2 | Build provider to delete files | Execute ./bin/build_provider |
3 | Initialize and apply Terraform | Run terraform init and terraform apply |
4 | Code block issue with < , related to JSON parsing |
Ensure the response is valid JSON and not HTML or other |
5 | Invalid character < , looking for value start |
Validate the data from the API is valid JSON |
6 | URL adjustment needed: hit “/api/home” instead | Change the URL to “/api/home” |
7 | Issue with endpoint in create script (only “home”) | Modify the endpoint to include “/api” in main.tf |
8 | Reattempt after endpoint correction | Rebuild, initialize, and apply |
I’ve found a better way to present the errors I encountered in these eight steps above, allowing you to tackle and resolve each one systematically.
The last apply will get your custom resource to be provisioned.
Great and cool!
NOTE: I had to deal with a 401 before my last succefull apply.
I fixed it with a simple replace of the current uuid with mine for the sinatra server.
State File Effects
It’s retaining this state, so we should be able to continue mocking it.
The state file in Terraform keeps track of the resource’s status, including the homeUUID
.
It’s crucial for Terraform to maintain resource mapping.
{
"version": 4,
"terraform_version": "1.6.0",
"serial": 3,
"lineage": "861def3e-f028-0a8f-c9bd-de51c87602fb",
"outputs": {},
"resources": [
{
"mode": "managed",
"type": "terratowns_home",
"name": "home",
"provider": "provider[\"local.providers/local/terratowns\"]",
"instances": [
{
"schema_version": 0,
"attributes": {
"content_version": 1,
"description": "Yaya is making something so great and so innovative.\nSomething Amazing is cooking with the greatest and only Yaya.\nFeeling Empowered.\n",
"domain_name": "veryveryrandomm.cloudfront.net",
"id": "833761b8-c2d7-4b17-91b4-0e2237184078",
"name": "How to play League in 2023",
"town": "gamers-grotto"
},
"sensitive_attributes": [],
"private": "bnVsbA=="
}
]
}
],
"check_results": null
}
Examine the state file to check for sensitive data (a valuable lesson to be learned).
- It does not store sensitive data here.
- We can view our schema for the resource and the provider we created.
The most important aspect is the ID, as the Terraform provider relies on it to maintain the mapping accurately.
Correct and Make Changes
You can make changes to your resource properties and apply them.
Terraform will detect and update the state accordingly.
Let’s Make Changes
- Change
name
toresource name
inmain.tf
and we encounter an error.- We wanted “home,” but it’s “House,” e.g., “HouseCreate.”
- Change
init
tofloat64
.- Another error occurs because of the state file.
- Error occurs because in one place it’s
init
, and in another, it’sfloat64
.
- Data type mapping:
- Create:
int
- Read:
float64
- Update:
int
- This mapping is based on how the response is received.
- Create:
- Resolution Steps:
- Build.
- Delete state.
- Stop Sinatra.
tf init
.tf apply --auto-approve
.- These steps are taken to resolve the error.
- It was created.
- Go to Sinatra and verify.
- Confirm that it’s created.
- Make a change, and then reapply.
- It detects that the description has changed and prompts for confirmation.
- Observe that the state now contains the updated information.
- There’s a double name in the update; change it to “description” and double-check.
- Ensure that there is no duplication.
- Make an update to the description or name.
- Run
tf apply
. - Ensure Sinatra is running.
Also, the apply do both update and show.
- Run
- It updates our state, and the names align as desired.
- Confirm that the state reflects the changes.
Cleanup
- Destroy the state. All is perfect!
Destroying the state will remove it completely. I used to play league of legends from the age 16 to 20.
Up from that point, I live in the cloud.
Considerations
- Are there any edge cases? (unknown)
- Are there any overlooked factors? (unknown)
Writing a robust Terraform provider can be a challenging task.
It’s impressive to achieve this, and it’s undeniably cool!
- The next step involves utilizing the provider with the actual Terratown endpoint.
- This will require setting up Terratown accounts and obtaining the necessary access.
We’ve reached a significant milestone with our progress.
🛑 | This is a hard good Stop! |
---|---|
We | did good and a lot. :D |
Target TerraTowns API
Welcome to 2.5.0
where we’ll be testing our custom provider against real TerraTowns infrastructure.
We will be configuring access tokens, endpoints, and other necessary settings to ensure our custom provider works seamlessly with TerraTowns.
Configuration Steps
We will get Terraformcloud back.
1. Access Token Setup
To use TerraTown, follow these steps:
- In your provider block, set the endpoint to
terratowns.cloud/api
. - Obtain your user UUID from your ExamPro account.
- Just there, click setting and
- Click Vending machine from left pane.
- Click on TerraTowns Access Code
- Get the access token below in the page.
Initial Terratowns Testing
- Go to your
main.tf
custom provider and fill in the uuid values.endpoint = "http://localhost:4567/api" user_uuid="" token=""
To;
endpoint = "http://terratowns.cloud/api" user_uuid="" token=""
⚠️Don’t commit your values. We will var them later.
- Fill with missingo as our first town of choice in our one and only
terratowns_home
resource.town = "gamers-grotto"
This how we target where our page will be part of.
- Try a random domain url for now. The domain doesn’t lead to any destination, but that’s fine. Let it be, for now. For You
- Execute the following command:
./bin/build_provider
- Initialize and apply Terraform:
tf init tf apply
After execution, you will receive valuable debugging information.
- Confirm by clicking
Yes.
You may have another 401 error, like we had earlier.- Map the UUID again in server.rb with the one from ExamPro.
- Make sure you target terraform.cloud in your endpoint.
- Make sure you apply your changes.
- Navigate to terratowns.cloud/t/ur-town.
You will find your profile displayed as a rectangle.
Clicking it will redirect you to the seemingly useless CloudFront.
CloudFront Mapping
Now, the question is whether the CloudFront real URL will take you to this location, or if we need to pass custom headers or headers through CloudFront to make it work. This will be determined.
Btw;
💡 | Update not allowed If you change your domain name |
---|---|
💡💡 | By design, You will need to perform a total teardown |
- To proceed, execute the following command:
tf destroy
- Update the url to map to your own cloudfront url.
- Click on your post in terratowns.cloud.
It is now directing to my page. It has anime in it with
Token Handling
To securely handle tokens, do the following:
- Avoid storing them directly in the
main.tf
file. - Use the
TF_VARS
environment variables for token management.Example for access token
export TF_VARS_terratowns_access_token="your_access_token_here" gp env TF_VARS_terratowns_access_token="your_access_token_here"
This will allow you to reference the access token in your main.tf
file using var.terratowns_access_token
.
Endpoint and UUID Configuration
- Set the TerraTowns endpoint in your
terraform.tfvars
file:terratowns_endpoint = "https://terratowns.cloud/api"
If set wrong; verify
TF_VAR
and notTF_VARS
.
You can access the endpoint in your code usingvar.terratowns_endpoint
.export TF_VAR_terratowns_endpoint="terratowns.cloud/api"
user_uuid
already there we can call it just var.user_uuid. Let employ this.. - Run the following commands in your gitpod and incldue your uuid;
export TF_VAR_teacherseat_user_uuid="pastehere" gp env TF_VAR_teacherseat_user_uuid="pastehere"
- Define those variables in your variables.tf, otherwise it wont work.
variable "terratowns_endpoint" {
type = string
}
variable "terratowns_access_token" {
type = string
}
variable "teacherseat_user_uuid" {
type = string
}
Second Sprint
- remove the previous
uuid
assignement. - Add it to the commented terrahouse module (for last tag)
- Employ the same for our access token.
export TF_VAR_terratowns_access_token="" gp env TF_VAR_terratowns_access_token=""
- Set your real Cloudfront URL and let’s give trry
domain_name = module.terrahouse_aws.cloudfront_url
- uncomment your terrahouse module.
- Update your
terraform.tfvars
fromterraform.tfvars.sample
just in case you have code inline. - Run
tf init
is success follow it bytf apply
;
Is there another error in the code?
Haha, remember that trap from Week Zero?
I fixed this already. But allow me please to fix yours in the next section.
This can help env | grep TF_VAR
Master Resolve For You
After initializing and applying Terraform configurations, you may encounter errors.
Ensure that the asset URL in tfvars.sample
ends with a /
. Also, add the asset_path
parameter to the TerraHouse AWS module.
- Append a
/
to the end of the asset URL intfvars.sample
. - Within the
terrrhouse_aws
module, specifically in the main Terraform root file main.tf, pass the value ofvar.assets._path
asasset_path
. - Feel free to run terraform apply which will end up a great success.
The sole distinctive aspect in this situation is related to the unique bucket name.
Bucket Name Randomization
I believe we’ve successfully handled validation and using a random provider, now it’s time to standardize the process for generating bucket names.
Let’s generate a random bucket name just for the sake of it.
To make the bucket name random:
- Obtain the bucket name from
tfvars
and the TerraHouse AWS module.module "terrahouse_aws" { source = "./modules/terrahouse_aws" user_uuid = var.teacherseat_user_uuid index_html_filepath = var.index_html_filepath error_html_filepath = var.error_html_filepath content_version = var.content_version assets_path = var.assets_path }
- Add comment for the bucket name setting in the CDN module and change as required;
comment = "Static website hosting for: ${var.bucket_name}" comment = "Static website hosting for: ${aws_s3_bucket.website_bucket.bucket}"
- Update variables and remove bucket validation in the CDN module. (I did in previous instruction)
- Modify the CDN module to reference the update bucket output.
name = "OAC ${aws_s3_bucket.website_bucket.bucket}" description = "Origin Access Controls for Static Website Hosting ${aws_s3_bucket.website_bucket.bucket}"
- The terraform plan touchpoint will tell that the bucket name it provides afterward (know after apply)
After applying Terraform configurations, the bucket name should be randomized.
The reason you observe the creation of only a single resource is because the apply becomes hang and it needed the AWS CLI to execute the invalidation process.
This was a preivously running workspace. I ran the aws cli again using our script briefly and get the apply going.
Testing Your First TerraHome
Now if you go you can find ur page there. terraform.cloud/
Check it out! |
---|
It is a mix of gaming and anime. Hope you like the creativity. |
Once you have configured TerraTown, we can:
- Access your TerraTown page.
- Configure multiple pages if needed.
- Test updates to ensure everything works as expected.
- Pushing our custom provider to the community.
For further, consider integrating TerraTown.. |
---|
with Terraform Cloud! |
We are currently in the final stages of completing version 2.5.0
and soon..
Everything😔
This specific provides a clear overview and step-by-step instructions for testing your custom provider against TerraTowns and pushing your post and your site to the town.
Terraform Turbocharge Workflow
Welcome to the Terraform 2.6.0
last official release with primary objective to integrate Terraform Cloud with our local development environments.
The following comes after an experience with different ways to work with terraform and thus coming up with the best possible solution.
The multiple home feature was part of this and now is part of 2.7.0
.
Learn more about our methodology.
Problematic 101
We previously operated exclusively using local execution, which resulted in rapid infrastructure planning and implementation.
❌ | We encountered a challenge |
---|---|
❌ | our infrastructure state was lost whenever we stopped our workspace |
Problematic 202
We have now made a complete shift to using Terraform Cloud, and this transition has yielded positive outcomes for our state management. Our infrastructure state is secure and preserved, ensuring that it remains intact even when we pause our work.
❌ | When utilizing Terraform Cloud’s compute resources |
---|---|
❌ | We have observed a decrease in execution speed |
Resolver Pro Max
These experiences bring us to the following solution.
Ensuring safety and persistence and never lose your state;
⛑️ | Preserve and manage our infrastructure state within Terraform Cloud |
---|---|
✅ | Use Terraform Cloud |
Optimize for the speed of our operations;
🏎️ | Conduct our computations and executions |
---|---|
✅ | using A local CDE infrastructure |
Welcome to this self-dedicated space for version 2.6.0
.
Head over Terraform Cloud
I thought you may forget what is that. Get the knowledge back.
- Access Terraform Cloud by following these steps:
- Click “Try Terraform Cloud.”
- Select your “terra-house-1” project.
- Configure Execution Mode:
- Change the execution mode to “local” in Terraform Cloud.
- After you picked your project, click settings from the left pane
The general pane will now popup.
- In the same page, swipe below and look for exec mode.
You will find that there is no option for local execution.
Also you see that your organization is set on the remote as shown in this asset.
- Click set to remote direction.
This will take you to organization general settings.
- Look at your org setting and leave it as it is.
- Go back to your project level.
- Click on custom, pick a the local option and click update below.
This is done to improve command execution speed as running commands directly in Terraform Cloud might be slower.
Remmember again that it’s essential to keep your state safe in the cloud.
Note: Make sure to update the project level settings, not the organization level.
- Return to GitPod:
- In your GitPod environment, add the “cloud” section in your main.tf file e.g.
cloud {
organization = "yayaintfcloud"
workspaces {
name = "terra-house-2023"
}
}
-
Build the plugin provider and set it to auto in the Terraform section.
-
Initialize and apply the Terraform configuration:
-
Run
terraform init
-
Followed by
terraform apply
to deploy your configuration to Terraform Cloud.
-
Observe the state is now in your terraform dotfile.
Have you also noticed? It is damn fast as it used to. But not only that..
- Monitor the Terraform Cloud dashboard and find your state; secure and downloadable, anytime and anywhere.
State is now managed by Terraform Cloud with your local execution to your development environment!
You can stop that and return to it later, knowing that your state is stored in the cloud.
The thing about Terraform Cloud that will not prompt for approvals but will display the current status of your operations.
Additional.
In order for me to do this.
I had to migrate to codespaces and did the entire process from the start to finish to name a few;
- Terraform Login
- AWS Credentials
- Configured terraformrc dot file.
- Explore the plugin path.
- Dealt with too many instant error/resolve.
Great, cool and easy now.
Consider pushing the provider to the registry for future use.
TerraHomes to TerraTowns
This is setup in 2.7.0
will lay the foundation to enable you with the creation of multiple homes or houses within the platform as it now supports up to twelve homes or houses.
- The First Music Post
- Check Out The Tunisian Food Post
- Terraform State Management
- Troubleshooting and Finalization
- Joyful conclusion
To enable the deployment of multiple homes simultaneously on the Terratowns platform, follow these steps:
Step 1: Create TerraHomes Folders
Create separate folders within the “public” directory for each home and migrate all project files to them.
- Create a folder named “home1” and move your project files into it.
- Create a second folder named “home2” and duplicate the files in the same format.
- Make sure the structure is something like this;
📁 public |─ 🏠 home1 | └─ 📂 assets | └─ 🖼️ great-stuff.png | └─ 📄 index.html | └─ 📄 error.html └─ 🏠 home2 └─ 📂 assets └─ 🖼️ great-stuff.png └─ 📄 index.html └─ 📄 error.html
Step 2: Rename Module
Since we are loving it on homes rather than houses..
- Rename the module from “terrahouse_aws” to
terrahome_aws.
Step 3: Update Configuration
Make the following updates to accommodate the changes related to terrahome_aws
variables and modules:
- Update the source to point to
./modules/terrahome_aws
instead of the old source to theterrahouse
.source = "./modules/terrahome_aws"
- Change the module name to match the theme, e.g.,
module "home_tnrap_hosting" {}
- Modify
output.tf
to referencehome_yourhousename_hosting
instead ofterrahouse_aws.
forbucket_name
value = module.terrahouse_aws.bucket_name
value = module.home_tnrap_hosting.bucket_name
value = module.home_tnfood_hosting.bucket_name
- Do the same in for the output of
s3_website_endpoint
value = module.terrahouse_aws.website_endpoint
value = module.home_tnrap_hosting.website_endpoint
value = module.home_tnfood_hosting.website_endpoint
- Do the same for the
cloudfront_url
and change output to referencedomain_name
instead.
value = module.terrahouse_aws.cloudfront_url
value = module.home_tnrap_hosting.domain_name
Step 4: Review and Update Variables in terraform.tfvars
Review and update variables in the module configurations. If you need to set nested Terraform variables in the tfvars file, refer to Terraform documentation for guidance.
- We asked GPT to help in this:
How do u set nested tf variables in tf vars file.
This is to make stuff look more damn neat.
- Remove previous
bucket_name
,index_html_filepath
,error_html_filepath
andcontent_version
absolute path.
index_html_filepath="/workspace/terraform-beginner-bootcamp-2023/public/index.html"
error_html_filepath="/workspace/terraform-beginner-bootcamp-2023/public/error.html"
assets_path="/workspace/terraform-beginner-bootcamp-2023/public/assets/"
content_version=1
- Update your
terraform.tfvars
andterraform.tfvars.sample
input for the public path and the content version in union like this;
house-name-1 = {
public_path = "/workspace/terraform-beginner-bootcamp-2023/public/tnrap"
content_version = 1
}
house-name-2 = {
public_path = "/workspace/terraform-beginner-bootcamp-2023/public/tnfood"
content_version = 1
}
Step 5: Modify Index Variables
- Remove previous
bucket_name
,index_html_filepath
,error_html_filepath
andcontent_version
variables definition.
Those in tfvars will be red inlined unless you perform the following task.
- Modify the variables to include references to public paths for your homes in your
variables.tf
file instead as nested.
variable "home-1" {
type = object({
public_path = string
content_version = number
})
}
.
.
.
variable "home-n" {
type = object({
public_path = string
content_version = number
})
}
Step 6: main.tf
root Remove Index and Error HTML References
Remove the “index_html_filepath” and “error_html_filepath” from your module block, and replace them with references to the “public_path” and “content_version” variables.
module "home_tnrap_hosting" {
source = "./modules/terrahome_aws"
user_uuid = var.teacherseat_user_uuid
public_path = var.arcanum.public_path
content_version = var.arcanum.content_version
}
Step 7: main.tf
root Update Resource Configuration
In the resource section, update the call with the new variables assigned to “cloudfront” and “content_version.”
resource "terratowns_home" "home" {
name = "How to play Arcanum in 2023!"
description = <<DESCRIPTION
DESCRIPTION
domain_name = module.home_arcanum_hosting.domain_name
town = ""
content_version = var.arcanum.content_version
}
Step 8: Storage Module Vars Update
- Start with output.tf change the output
"cloudfront_url"
to"domain_name"
- In
resource-storage.tf
change thesource
andetag
path forindex_html
,error_html
foraws_s3_object
to map to our new vars.source = var.index_html_filepath source = "${var.public_path}/index.html" etag = filemd5(var.index_html_filepath) etag = filemd5("${var.public_path}/index.html")
Apply the same for the error file.
source = var.error_html_filepath source = "${var.public_path}/error.html" etag = filemd5(var.error_html_filepath) etag = filemd5("${var.public_path}/error.html")
- In the same file update source, etag but also the
for_each
forupload_assets
.
for_each = fileset(var.assets_path,"*.{jpg,png,gif}")
for_each = fileset("${var.public_path}/assets","*.{jpg,png,gif}")
source = "${var.assets_path}/${each.key}"
etag = filemd5("${var.assets_path}${each.key}")
source = "${var.public_path}/assets/${each.key}"
etag = filemd5("${var.public_path}/assets/${each.key}")
Step 9: Module variables.tf
Updates
- change our
index_html_filepath
variable topublic_path
instead.variable "public_path" {}
- Remove its validation and rename its description.
description = "The file path for the public directory"
- Leave it type string as it is.
type = string
- in the same file, terminate both
assets_path
anderror_html_filepath
and its validation.
Step 10: Duplicate Step 5-6-7 for Another Home
Duplicate the entire process for the module/resource to create a new home just below it.
module "home_tnfood_hosting" {
source = "./modules/terrahome_aws"
user_uuid = var.teacherseat_user_uuid
public_path = var.tnfood.public_path
content_version = var.tnfood.content_version
}
resource "terratowns_home" "home_tnfood" {
name = "Showing you our Tunisian Food"
description = <<DESCRIPTION
Add yours here!
DESCRIPTION
domain_name = module.home_tnfood_hosting.domain_name
town = "missingo"
content_version = var.tnfood.content_version
}
Step 9: Build and Deploy
To deploy your changes, follow these steps:
- Ensure you are in the project directory, go to .gitpod.yml
- add the following command with source below the copy command:
cp $PROJECT_ROOT/terraform.tfvars.example $PROJECT_ROOT/terraform.tfvars source ./bin/build_provider
- Run tf init to initialize your infra and new resources.
- Run
tf plan
, observe and runtf apply
.
With these changes, you should be able to see both pages listed in your Terraform Cloud dashboard.
The First Music Post
I listen to music every day, mostly diverse genres. These are my original music choices, and you can expect me to listen to them with neutral to somewhat excited energy.
Check out The Music Home
You can listen even without signing in to Spotify. I’m confident you’ll feel a closer connection if you give one of them a listen.
Check Out The Tunisian Food Post
We’re a country that truly cherishes its food culture. It plays a significant role in our lives.
Fortunately, I’m often occupied with computer work or at the gym;
otherwise, I might end up gaining weight.
Check Out The list in TerraHome
These dishes are worth the extra calories; give them a try and keep me in mind.
Terraform State Management
- Execute
tf state list
to view the resources in Terraform Cloud within Gitpod. - Perform
tf state pull
to get the latest from tfcloud.
Troubleshooting and Finalization
- Address any errors encountered during the plan and apply phases.
- Resolve undefined index, etag path, or other issues.
- Update outputs to match changes made in the configuration.
- Ensure the plan and apply phases complete successfully.
- Confirm changes in the Terraform Cloud interface.
- Delete the Terraform Cloud resources if necessary.
Joyful conclusion
Version 2.7.0
originally 2.6.0
of the project focuses on integrating Terraform Cloud, enabling the creation of multiple homes or houses, and ensuring efficient configuration management.
Show your creativity.
Make more homes!
TerraTowns Potential Unlocked
In the earlier release tagged as 2.7.0
, originally conceived as 2.6.0
, I decided to refine the curriculum for your benefit and your understanding.
2.6.0
— Work with terraform cloud and local execution.2.7.0
— Create Two TerraHomes and adhrere to the House Theme.2.7.1
— Create Three More Homes and Target All TerraTowns.
Today, our aim is to address the remaining three endpoints, unlocking the full potential of Terratown.
We end up with a total of six projects nested within our /public
repository.
This Incl. 1 Mixer town for testing in Missingo.
📁./Terraformer
|─ 🏠 favshow
| └─ 📂 assets
| | └─ 🖼️ 26-july-2018-bb-ban.png
| | └─ 🖼️ august-18-2018.jpeg
| | └─ 🖼️ bb-26-july-2018.png
| | └─ 🖼️ br-ba-logo.png
| └─ 📄 error.html
| └─ 📄 index.html
|─ 🏠 lolplayz
| └─ 📂 assets
| | └─ 🖼️ 2019-got-serious-to-college.jpeg
| | └─ 🖼️ favi.png
| | └─ 🖼️ gamer-ERR.png
| | └─ 🖼️ gamer-proof.jpeg
| | └─ 🖼️ gamer-webpage-somejs.png
| | └─ 🖼️ gold-lol-18-aug-2018.jpeg
| | └─ 🖼️ lol-chall.jpeg
| | └─ 🖼️ old-time.jpeg
| └─ 📄 error.html
| └─ 📄 index.html
|─ 🏠 mixer
| └─ 📂 assets
| | └─ 🖼️ elizabeth-7-deadly-sins.png
| | └─ 🖼️ goku.jpg
| └─ 📄 error.html
| └─ 📄 index.html
| └─ 📄 oldindex.html
|─ 🏠 tnfood
| └─ 📂 assets
| | └─ 🖼️ bambalouni.jpg
| | └─ 🖼️ couscous.jpg
| | └─ 🖼️ food-error-page.png
| | └─ 🖼️ food-terrahome.png
| | └─ 🖼️ lablebi.jpg
| | └─ 🖼️ LA-MLOUKHIA.png
| | └─ 🖼️ makroud.jpg
| | └─ 🖼️ slata-mechouia.jpg
| | └─ 🖼️ tunisian-food-masfouf.jpg
| └─ 📄 error.html
| └─ 📄 index.html
|─ 🏠 tnrap
| └─ 📂 assets
| | └─ 🖼️ loading-lazy.png
| | └─ 🖼️ music-terratown.png
| | └─ 🖼️ tn-music-errored.png
| └─ 📄 error.html
| └─ 📄 index.html
└─ 🏠 travlz
└─ 📂 assets
| └─ 🖼️ el-jem.jpg
| └─ 🖼️ medina.jpg
| └─ 🖼️ moss.jpg
| └─ 🖼️ musee-bardo.JPG
| └─ 🖼️ sidi.jpg
| └─ 🖼️ sousse-ribat.JPG
| └─ 🖼️ tn.png
| └─ 🖼️ travel-err.png
| └─ 🖼️ travel-page.png
| └─ 🖼️ tunis.jpg
| └─ 🖼️ zitounaaa.jpg
└─ 📄 error.html
└─ 📄 index.html
I ensured that these homes reflect my personality, allowing me to reveal more about who I am as a great person.
TerraHome: Coding a New Home
In the following section, we will cover the steps necessary for you to add a new home.
Although we’ve already covered this process in the previous version 2.7.0, we will do it again to assure we know what we are doing.
Instructions assume that you have configured your variables at the module level |
---|
Here |
- Create a new TerraHome in the
public
directory.
I can’t instruct you further what you are allowed to create inside.
I end up following the rules creating html files and assets.
Feel free to break them please.
Do more.
When your projects are in place. Follow this configuration as code.
- Create a module in your main.tf with the home name e.g.
module "home_change-this_hosting" {}
- Go define your Home variables in
variables.tf
for the new home.
variable "change-this"
{
type = object
(
{
public_path = string
content_version = number
}
)
}
- Set the actual content of your new home variables in your
terraform.tfvars
change-this = {
public_path = "/workspace/terraform-beginner-bootcamp-2023/public/change-this"
content_version = 1
}
- Go back to your module in
main.tf
and configure yourpublic_path
andcontent_version
with these variables.
module "home_change-this_hosting" {
source = "./modules/terrahome_aws"
user_uuid = var.teacherseat_user_uuid
public_path = var.change-this.public_path
content_version = var.change-this.content_version
}
- To make sure your new home variables are safe, copy the content from your
terraform.tfvars
toterraform.tfvars.sample
Remmber we have a cp command to get that content.
That it with the configuration.
More homes? Read this again.
TerraHomes Delivery
Terraform configured? Project Files correctly tested? Ready to launch.
- Run the
build_provider
script to get the binairy. - Make sure you sinatra server is running, if not run
bundle install bundle exec ruby server.rb
- Terraform init and make sure you are authenticated to terraform cloud, if not run and get that token.
terraform login
terraform plan
to see the great stuff coming in.terraform apply -lock=false
in cases your state is locked in tf cloud.
- The existing homes will be updated if any updates are available.
- The new homes can be added with that single command.
- The hard part is always building the infra like a symphony.
TerraTowns Deliverables
As I mentioned before, I designed TerraHomes for every TerraTown.
This section serves as a showcase of the most recent three additions.
→ To learn more about the methodology.
Breaking Bad FanPage
This page is my way of expressing my admiration for the TV show. I’ve incorporated a quote generator and included assets that hold sentimental value from a dear friend.
The post Title and description specified;
name = "The Best TV Show You'll Ever Watch"
description = <<DESCRIPTION
💥Join us as we unravel the gripping tale of high-stakes crime, morality, and transformation that has captured the hearts of millions worldwide.💥
DESCRIPTION
TerraTown Post Show Up
The Project TerraHome
Internal URL |
---|
CloudFront URL |
Past Gaming Passion
This project is a highlight to what I used to play a lot in the past. It gives some insights to why I am good in english? And why I find mysef a master in the cloud?
The post Title and description specified;
name = "What Yahya Used To play A lot?"
description = <<DESCRIPTION
Uncover Yahya's past gaming obsession and how his extensive involvement with League of Legends eventually severed that connection, enabling him to devote more time to other pursuits.
DESCRIPTION
TerraTown Post Show Up
The Project TerraHome
Internal URL | CloudFront URL |
---|---|
Visit Tunisia Website
Our economy is a bit messed up but we have great places.
I think a smart person can visit have lots of fun with dead cheap money and go back happy.
The post Title and description specified;
name = "The Best TV Show You'll Ever Watch"
description = <<DESCRIPTION
💥Join us as we unravel the gripping tale of high-stakes crime, morality, and transformation that has captured the hearts of millions worldwide.💥
DESCRIPTION
}
TerraTown Post Show Up
The Project TerraHome
Internal URL | CloudFront URL |
---|---|
Considerations
Unsupported Image Format (JPEG):
Corrected the issue where JPEG images were not supported.
Image Format Conversion (JPEG to PNG):
Converted the existing JPEG images to PNG format to ensure compatibility with the project.
File Extension Renaming (JPG to jpg):
Modified file extensions from “JPG” to “jpg” for consistent and standardized naming.
User Interface and Slider Enhancements:
Made necessary fixes and improvements to the user interface and slider functionality to enhance the project’s overall usability and appearance.
Wrapping Up
In case you weren’t aware, all the strategies I used have been revealed
(Far surpassing those below—Dig in🕳️).
My relentless pursuit is your advantage ❤️
My Gifts | Description |
---|---|
Chats | Learn how I interact with this thing to get things done. |
Trees | Explore the beauty of code with this cool Tree trick. |
ExamPro | Captured a collection of assets thanks to ExamPro. |
Issue 69 | Learn the project next steps and the number sixty nine. |
Creativity knows no bounds, and in the absence of drafts, the untamed mind finds its truest expression.
Measured Finish
This is the bootcamp seen and constructed through the eyes of the author.
Experiences and methods to reach the top varies.
ℹ️: The project concluded with 31 in commits, tags, branches and issues |
The author has absolute confidence in the enormous potential of this journey for you and his commitment to your triumph remains unswerving and ever-expanding.
Technical Reference
Below are the bootcamp projects, each accessible via a subdomain endpoint.
https://missingo.ya-ya.tech
https://cooker-cove.ya-ya.tech
https://melomaniac-mansion.ya-ya.tech
https://gamers-grotto.ya-ya.tech
https://the-nomad-pad.ya-ya.tech
https://video-valley.ya-ya.tech
The information mentioned above has been elaborated in issue number sixty-nine.
Surpassing All Endings Expectations
The bootcamp concluded,
The author proudly received both tangible and moral credentials from the organizers.
He looks forward to sharing additional work with you enthusiastically.
↓ You enjoyed this cute little bootcamp?
I did a bigger one.
And you may as well like to abstract my work.
See you in the cloud,