MIT on Tom Webster

Software Engineering Essentials: Flags

Thu, 14 Nov 2024 00:00:00 -0600

Software Engineering Essentials is a series of blog posts designed to help you get started with a wide variety of software engineering topics. This post was originally part of my Go-Twitter project, a project-based curriculum designed to take someone from “zero” to “competent” in the world of software engineering.

All writing for this post is licensed under CC-BY-4.0, while all code is licensed under the MIT license.

Flags

Flag Usage

Flags are effectively named arguments, with some extra features. Instead of needing to remember the order of your program’s arguments, you can just specify the name of the flag and what that variable should be set to. Let’s take a look at a simple example:

01-simple-flags/main.go:

package main

import (
	"flag"
	"fmt"
)

func main() {
	greeting := flag.String("greeting", "", "")
	name := flag.String("name", "", "")
	flag.Parse()
	fmt.Printf("%s, %s!\\n", *greeting, *name)
}

Now, if we run this program:

go run example-code/02-command-line-arguments-and-flags/02-flags/01-simple-flags/main.go
 !

Huh, that looks weird. This is because our greeting and name variables are empty. We never specified the flags. How do we know what the flags are? Flags have built-in help functionality that will display all of the options available. Most programs use -h or --help to display program usage text, including flags and arguments.

go run example-code/02-command-line-arguments-and-flags/02-flags/01-simple-flags/main.go -h
Usage of /tmp/go-build1913891522/b001/exe/main:
 -greeting string

 -name string

So now we can see that this program accepts the -greeting and -name flags. Let’s set some values:

go run example-code/02-command-line-arguments-and-flags/02-flags/01-simple-flags/main.go -greeting="Sup" -name Tom
Sup, Tom!

Flag arguments can be set with some optional grammar to help avoid problems with ambiguity or spaces:

flag value: For one-word values
- Unquoted spaces can change how your command is interpreted, leading to missing values and skipped flags. See the appendix section How Spaces Can Affect Flags for an example.
flag "value with spaces or special characters": For multi-word values or values that contain some special characters.
- If your value has a $ symbol, you may need to use single-quotes (') to avoid the shell interpreting $ as the start of a shell variable.
flag="value": This is the belt-and-suspenders of flag declaration.
- The = just removes space from the declaration and makes it very clear that this particular value belongs to that particular flag. This can be helpful to aid in readability, especially if you’re writing out examples or documentation.

We can also specify the flags in any order:

go run example-code/02-command-line-arguments-and-flags/02-flags/01-simple-flags/main.go -name="General" -greeting="Hello There"
Hello There, General!

Or not specify a value for -name at all!

go run example-code/02-command-line-arguments-and-flags/02-flags/01-simple-flags/main.go -greeting="Hello There"
Hello There, !

We can even abuse the flags to make the program finish the meme:

go run example-code/02-command-line-arguments-and-flags/02-flags/01-simple-flags/main.go -greeting="General Kenobi\\!" -name="You are a bold one"
General Kenobi!, You are a bold one!

Defaults

Let’s re-create the program that inserted profiles into a database from the end of the Command Line Arguments chapter, but with flags instead:

02-profile-saver/main.go:

package main

import (
	"flag"
	"fmt"
)

func main() {
	firstName := flag.String("first-name", "", "")
	lastName := flag.String("last-name", "", "")
	favFruit := flag.String("fruit", "", "")
	favVeg := flag.String("veg", "", "")
	favPasta := flag.String("pasta", "", "")
	favMeal := flag.String("meal", "", "")
	riceOrBeans := flag.String("rice-or-beans", "", "")
	favDesert := flag.String("desert", "", "")
	favDessert := flag.String("dessert", "", "")
	flag.Parse()

	fmt.Printf(`Profile Name: %s %s
	Rice or Beans: %s
	Favorites:
		Fruit: %s
		Vegetable: %s
		Pasta: %s
		Desert: %s
		Dessert: %s
		Meal: %s
`, *firstName, *lastName, *riceOrBeans, *favFruit, *favVeg, *favPasta, *favDesert, *favDessert, *favMeal)
}

Right now if we run this program with no flags set, we get this results:

go run example-code/02-command-line-arguments-and-flags/02-flags/02-profile-saver/main.go
Profile Name:
 Rice or Beans:
 Favorites:
 Fruit:
 Vegetable:
 Pasta:
 Desert:
 Dessert:
 Meal:

Lots of blank spaces here. We don’t want to commit empty values to the database, it can introduce ambiguity. Did this person not have a favorite pasta or did we just forget to add the data? Being explicit in programming has several benefits, including clarity-of-intent, so let’s add some defaults.

Default values can provide the usual options or placeholder data. Flags make this easy. For now, let’s just use UNKNOWN as the profile placeholder. With the stanadrd flag package, setting defaults is easy. Let’s take a look at the GoDocs for flag.String, available here:

func String
func String(name string, value string, usage string) *string
String defines a string flag with specified name, default value, and usage string. The return value is the address of a string variable that stores the value of the flag.

So the second function argument allows us to specify a default value for that particular flag. Let’s do that now:

02-profile-saver/main.go:

package main

import (
	"flag"
	"fmt"
)

func main() {
	firstName := flag.String("first-name", "UNKNOWN", "")
	lastName := flag.String("last-name", "UNKNOWN", "")
	favFruit := flag.String("fruit", "UNKNOWN", "")
	favVeg := flag.String("veg", "UNKNOWN", "")
	favPasta := flag.String("pasta", "UNKNOWN", "")
	favMeal := flag.String("meal", "UNKNOWN", "")
	riceOrBeans := flag.String("rice-or-beans", "UNKNOWN", "")
	favDesert := flag.String("desert", "UNKNOWN", "")
	favDessert := flag.String("dessert", "UNKNOWN", "")
	flag.Parse()

	fmt.Printf(`Profile Name: %s %s
	Rice or Beans: %s
	Favorites:
		Fruit: %s
		Vegetable: %s
		Pasta: %s
		Desert: %s
		Dessert: %s
		Meal: %s
`, *firstName, *lastName, *riceOrBeans, *favFruit, *favVeg, *favPasta, *favDesert, *favDessert, *favMeal)
}

Now when we run this program without any flags set:

go run example-code/02-command-line-arguments-and-flags/02-flags/02-profile-saver/main.go
Profile Name: UNKNOWN UNKNOWN
 Rice or Beans: UNKNOWN
 Favorites:
 Fruit: UNKNOWN
 Vegetable: UNKNOWN
 Pasta: UNKNOWN
 Desert: UNKNOWN
 Dessert: UNKNOWN
 Meal: UNKNOWN

There we go, that at least looks more complete.

Now let’s actually use those flags to set some values… but wait, what were the flags again? Let’s use -h to see what options we have:

go run example-code/02-command-line-arguments-and-flags/02-flags/02-profile-saver/main.go -h
Usage of /tmp/go-build261536590/b001/exe/main:
 -desert string
 (default "UNKNOWN")
 -dessert string
 (default "UNKNOWN")
 -first-name string
 (default "UNKNOWN")
 -fruit string
 (default "UNKNOWN")
 -last-name string
 (default "UNKNOWN")
 -meal string
 (default "UNKNOWN")
 -pasta string
 (default "UNKNOWN")
 -rice-or-beans string
 (default "UNKNOWN")
 -veg string
 (default "UNKNOWN")

Ah, got it. Now let’s actually use these:

go run example-code/02-command-line-arguments-and-flags/02-flags/02-profile-saver/main.go -first-name Bob -last-name Sacamano -fruit Apple -veg "Green Beans" -pasta Manicotti -meal Steak -desert Sahara -dessert Sundae -ri
ce-or-beans Rice
Profile Name: Bob Sacamano
 Rice or Beans: Rice
 Favorites:
 Fruit: Apple
 Vegetable: Green Beans
 Pasta: Manicotti
 Desert: Sahara
 Dessert: Sundae
 Meal: Steak

So now we’re able to print out all of the profile information using flags. Its still a lot of flags to remember, but at least we have the default help text and the order of options doesn’t matter.

Help Text

As we saw above, -h is pretty helpful to understand which flags are available, what type is expected, and what the defaults are, but we can add some more context through help text. As always, reading the documentation can help us find the right place to add help text to our flags. For now, let’s just add one piece of help text and see how it compares to the rest of the flags:

02-profile-saver/main.go:

package main

import (
	"flag"
	"fmt"
)

func main() {
	firstName := flag.String("first-name", "UNKNOWN", "the first name of the user profile")
	lastName := flag.String("last-name", "UNKNOWN", "")
	favFruit := flag.String("fruit", "UNKNOWN", "")
	favVeg := flag.String("veg", "UNKNOWN", "")
	favPasta := flag.String("pasta", "UNKNOWN", "")
	favMeal := flag.String("meal", "UNKNOWN", "")
	riceOrBeans := flag.String("rice-or-beans", "UNKNOWN", "")
	favDesert := flag.String("desert", "UNKNOWN", "")
	favDessert := flag.String("dessert", "UNKNOWN", "")
	flag.Parse()

	fmt.Printf(`Profile Name: %s %s
	Rice or Beans: %s
	Favorites:
		Fruit: %s
		Vegetable: %s
		Pasta: %s
		Desert: %s
		Dessert: %s
		Meal: %s
`, *firstName, *lastName, *riceOrBeans, *favFruit, *favVeg, *favPasta, *favDesert, *favDessert, *favMeal)
}

Now let’s check out the help text:

go run example-code/02-command-line-arguments-and-flags/02-flags/02-profile-saver/main.go -h
Usage of /tmp/go-build1166983742/b001/exe/main:
 -desert string
 (default "UNKNOWN")
 -dessert string
 (default "UNKNOWN")
 -first-name string
 the first name of the user profile (default "UNKNOWN")
 -fruit string
 (default "UNKNOWN")
 -last-name string
 (default "UNKNOWN")
 -meal string
 (default "UNKNOWN")
 -pasta string
 (default "UNKNOWN")
 -rice-or-beans string
 (default "UNKNOWN")
 -veg string
 (default "UNKNOWN")

Neat! Now let’s add the rest:

02-profile-saver/main.go:

package main

import (
	"flag"
	"fmt"
)

func main() {
	firstName := flag.String("first-name", "UNKNOWN", "the first name of the user profile")
	lastName := flag.String("last-name", "UNKNOWN", "the last name of the user profile")
	favFruit := flag.String("fruit", "UNKNOWN", "the user's favorite fruit")
	favVeg := flag.String("veg", "UNKNOWN", "the user's favorite vegetable")
	favPasta := flag.String("pasta", "UNKNOWN", "the user's favorite type of pasta")
	favMeal := flag.String("meal", "UNKNOWN", "the user's all-time favorite meal")
	riceOrBeans := flag.String("rice-or-beans", "UNKNOWN", "whether the user prefers a side of rice or beans")
	favDesert := flag.String("desert", "UNKNOWN", "the user's favorite desert (usually hot and sandy)")
	favDessert := flag.String("dessert", "UNKNOWN", "the user's favorite dessert (usually a sweet after-meal treat)")
	flag.Parse()

	fmt.Printf(`Profile Name: %s %s
	Rice or Beans: %s
	Favorites:
		Fruit: %s
		Vegetable: %s
		Pasta: %s
		Desert: %s
		Dessert: %s
		Meal: %s
`, *firstName, *lastName, *riceOrBeans, *favFruit, *favVeg, *favPasta, *favDesert, *favDessert, *favMeal)
}

And if we check out help text:

go run example-code/02-command-line-arguments-and-flags/02-flags/02-profile-saver/main.go -h
Usage of /tmp/go-build2677080627/b001/exe/main:
 -desert string
 the user's favorite desert (usually hot and sandy) (default "UNKNOWN")
 -dessert string
 the user's favorite dessert (usually a sweet after-meal treat) (default "UNKNOWN")
 -first-name string
 the first name of the user profile (default "UNKNOWN")
 -fruit string
 the user's favorite fruit (default "UNKNOWN")
 -last-name string
 the last name of the user profile (default "UNKNOWN")
 -meal string
 the user's all-time favorite meal (default "UNKNOWN")
 -pasta string
 the user's favorite type of pasta (default "UNKNOWN")
 -rice-or-beans string
 whether the user prefers a side of rice or beans (default "UNKNOWN")
 -veg string
 the user's favorite vegetable (default "UNKNOWN")

So much better. Now our program has clear flagged arguments, defaults set, and some help text to explain what each flag is intended for. However, this could be cleaned up a bit more. To clean up the help text, let’s dig into constants.

Constants

Constants work very similarly to variables, but with one critical difference: They cannot be changed at runtime. Constants are set in your source code and compiled-in, after that, the value cannot be changed. Constants are helpful to document and organize variables that control large pieces of your application or that are used in a variety of places, but the values won’t change.

Let’s dig in with an example. Let’s say you were making a program that talks to a website to get information. The functions you write to talk to the website are all going to use the same address over and over and over again. Instead of typing https://www.example.com over and over again, you can put this address in a constant declaration near the top of your file, then just reference it using the constant name:

package main

import (
	"fmt"
)

const (
 website = "<https://www.example.com>"
)

func main() {
 fmt.Println(website)
}

So constants work kinda like variables, but let’s see what happens when we try to change the value in our main function:

package main

import (
	"fmt"
)

const (
	website = "<https://www.example.com>"
)

func main() {
	fmt.Println(website)
	website = "A new value"
	fmt.Println(website)
}

When we run it, we get this:

main.go:13:2: cannot assign to website (untyped string constant "<https://www.example.com>")

The constants must remain constant. Trying to change them results in the program refusing to build.

So, constants can be useful to help avoid re-typing, but they can also help generally clean up your code. Right now in our profile-saver program, we have a lot of loose help text that we can clean up by moving those help strings from the flag.String functions into a constant declaration. So we can turn this:

func main() {
	firstName := flag.String("first-name", "UNKNOWN", "the first name of the user profile")
	lastName := flag.String("last-name", "UNKNOWN", "the last name of the user profile")
	favFruit := flag.String("fruit", "UNKNOWN", "the user's favorite fruit")
	favVeg := flag.String("veg", "UNKNOWN", "the user's favorite vegetable")
	favPasta := flag.String("pasta", "UNKNOWN", "the user's favorite type of pasta")
	favMeal := flag.String("meal", "UNKNOWN", "the user's all-time favorite meal")
	riceOrBeans := flag.String("rice-or-beans", "UNKNOWN", "whether the user prefers a side of rice or beans")
	favDesert := flag.String("desert", "UNKNOWN", "the user's favorite desert (usually hot and sandy)")
	favDessert := flag.String("dessert", "UNKNOWN", "the user's favorite dessert (usually a sweet after-meal treat)")
...

Into this:

const (
	firstNameHelp = "the first name of the user profile"
	lastNameHelp = "the last name of the user profile"
	fruitHelp = "the user's favorite fruit"
	vegHelp = "the user's favorite vegetable"
	pastaHelp = "the user's favorite type of pasta"
	mealHelp = "the user's all-time favorite meal"
	riceOrBeansHelp = "whether the user prefers a side of rice or beans"
	desertHelp = "the user's favorite desert (usually hot and sandy)"
	dessertHelp = "the user's favorite dessert (usually a sweet after-meal treat)"
)

func main() {
	firstName := flag.String("first-name", "UNKNOWN", firstNameHelp)
	lastName := flag.String("last-name", "UNKNOWN", lastNameHelp)
	favFruit := flag.String("fruit", "UNKNOWN", fruitHelp)
	favVeg := flag.String("veg", "UNKNOWN", vegHelp)
	favPasta := flag.String("pasta", "UNKNOWN", pastaHelp)
	favMeal := flag.String("meal", "UNKNOWN", mealHelp)
	riceOrBeans := flag.String("rice-or-beans", "UNKNOWN", riceOrBeansHelp)
	favDesert := flag.String("desert", "UNKNOWN", desertHelp)
	favDessert := flag.String("dessert", "UNKNOWN", dessertHelp)
...

There’s still a lot of text, but its a bit more organized and easier to work with. This is one of the nicest bonus features of constants: They make great organizational tools. If you have certain aspects of your program that can be tuned by yourself or another developer, it may be a good idea to put them in a const declaration at the top of the file so they’re easy to access and conveniently grouped.

Its worth keeping in mind that not everything can be a constant. Due to the way the Go compiler works, complex expressions cannot be pre-evaluated. Said another way: Constants are baked into your application when you run go build. According to the Go language spec, the only valid constant types are:

Runes (characters)
Strings
Booleans (true or false)
Numeric types (ints, floats, etc…)

If you want the same style of organization for types that don’t work as constants, you can use a var declaration instead:

var (
	aMap = map[int]string{
		1: "one",
		2: "two",
		3: "three",
	}
)

Appendix and References

References

Appendix

How Spaces Can Affect Flags

go run example-code/02-command-line-arguments-and-flags/02-flags/01-simple-flags/main.go -greeting Hello There -name General
Hello, !

go run example-code/02-command-line-arguments-and-flags/02-flags/01-simple-flags/main.go -greeting "Hello There" -name General
Hello There, General!

go run example-code/02-command-line-arguments-and-flags/02-flags/01-simple-flags/main.go -greeting="Hello There" -name General
Hello There, General!

Software Engineering Essentials: Command Line Arguments

Wed, 13 Nov 2024 00:00:00 -0600

All writing for this post is licensed under CC-BY-4.0, while all code is licensed under the MIT license.

Command Line Arguments

What are command line arguments?

Command line arguments are one way to pass user input into your application. Let’s take a look at the common Unix utility ls. ls is a small application that lists files and directories, just like double-clicking on a folder in Windows Explorer, ls allows you to peer into directories and get additional information about files, like ownership information, size, last modification timestamps, and more. ls, like many command line applications, can take in arguments:

ls some-directory

In this example ls is the application you are running and some-directory is the argument. What you’re asking ls to do is give you a listing of all files and folders inside of some-directory.

How can my program use arguments?

Because programming is generally open-ended, you can utilize command line arguments to control any aspect of your program. You could use an argument to control which IP address and port your application listens for requests on, or what to print, or if an operation should target your test or production environments. Let’s take a look at a simple example:

Here’s a simple Go program that will print a person’s name and age on the command line:

01-simple-arguments/main.go:

package main

import (
 "fmt"
 "os"
)

func main() {
 fmt.Printf("My name is %s and my age is %s.", os.Args[1], os.Args[2])
}

So if we run go run example-code/02-command-line-arguments-and-flags/01-command-line-arguments/01-simple-arguments/main.go Bob 99, we’ll see: My name is Bob and my age is 99.

In this example, we use the os package to get an array of arguments provided on the command line. Arrays in Go start at 0, so we’re actually printing the second and third argument. The first argument is the location of the program we’re running. Since we’re using go run, it will be in a temporary, auto-generated location. Let’s add this block of code to our program and look at each argument individually:

fmt.Println("Our program's arguments:")
for i, arg := range os.Args {
 fmt.Printf("os.Args[%d]: '%s'\\n", i, arg)
}

This code will print out each argument and its number:

Our program's arguments:
os.Args[0]: '/tmp/go-build201851007/b001/exe/main'
os.Args[1]: 'Bob'
os.Args[2]: '99'

So /tmp/go-build201851007/b001/exe/main is the temporary compiled version of our program, made by the go run command. The other arguments, Bob and 99 are the arguments we provided on the command line.

So, what happens if we provide less than 2 arguments to our program?

go run example-code/02-command-line-arguments-and-flags/01-command-line-arguments/01-simple-arguments/main.go Bob
panic: runtime error: index out of range [2] with length 2

goroutine 1 [running]:
main.main()
 /home/samurailink3/git/go-twitter/example-code/02-command-line-arguments-and-flags/01-command-line-arguments/01-simple-arguments/main.go:9 +0x21c
exit status 2

Uh oh… panic: runtime error: index out of range [2] with length 2… That doesn’t look good. What happened is we tried to access a variable that was outside of the array. Here’s a simple diagram of the os.Args array and what we’re trying to access:

 Arg 0: Our program Arg 1: Our provided name Arg 2: What we're trying to access
 | | |
 v v v
[ "/tmp/go-build201851007/b001/exe/main", "Bob" ]

The os.Args array only contains 2 items (os.Args[0] and os.Args[1]), but we’re trying to access something in the third place (os.Args[2]), beyond the end of the array. This is an issue, so the code panics. How can we defend against that? We want to avoid panicing and let the user know they’re missing a required field.

The len check:

if len(os.Args) < 3 {
 // Do something here
}

We can get the number of items present in the os.Args array and do something if we don’t have enough. First, let’s print out a helpful message for the user:

if len(os.Args) < 3 {
 fmt.Println("This program requires two arguments: Name and Age, you have", len(os.Args)-1)
}

When we run this with only the first argument, we get:

go run example-code/02-command-line-arguments-and-flags/01-command-line-arguments/01-simple-arguments/main.go Bob
This program requires two arguments: Name and Age, you have 1
panic: runtime error: index out of range [2] with length 2

goroutine 1 [running]:
main.main()
 /home/samurailink3/git/go-twitter/example-code/02-command-line-arguments-and-flags/01-command-line-arguments/01-simple-arguments/main.go:13 +0x29b
exit status 2

Generally, we don’t want to show panics to the user, so let’s exit with a non-successful error code (to learn more about error codes, check out the Advanced Bash-Scripting Guide):

	if len(os.Args) < 3 {
		fmt.Println("This program requires two arguments: Name and Age, you have", len(os.Args)-1)
		// See "Advanced Bash-Scripting Guide: Exit Codes With Special Meanings"
		// in the References section for why we use "2" here.
		os.Exit(2)
	}

Now when we run our program with the missing argument, we get:

go run example-code/02-command-line-arguments-and-flags/01-command-line-arguments/01-simple-arguments/main.go Bob
This program requires two arguments: Name and Age, you have 1
exit status 2

That’s way cleaner. Now if we run it with the right number of arguments, we see:

go run example-code/02-command-line-arguments-and-flags/01-command-line-arguments/01-simple-arguments/main.go Bob 99
My name is Bob and my age is 99.
Our program's arguments:
os.Args[0]: '/tmp/go-build1771918657/b001/exe/main'
os.Args[1]: 'Bob'
os.Args[2]: '99'

When should I use arguments?

Arguments are best used when an application has a limited number of distinct options or your program operates on an unlimited number of the same argument. Let’s expand that previous statement:

So git is a great example of having a limited (and segmented) number of distinct options. It can do a lot, but the options are segmented by sub-command. So, something git branch would have pretty simple arguments, like git branch -D branch-name and git branch --track local-branch remote-branch. There are a couple of arguments to keep in your head, not too bad.

Another program we talked about earlier, ls, is a great example of having an unlimited number of the same argument. So you can run ls with zero arguments, which just shows the files and folders present in your current directory, but we can also run ls some-directory and-another-one but-also-a-third to see what’s inside those three distinct directories. ls supports an unlimited number of arguments, but each one does the same thing, so its not hard to remember or think about.

Let’s say you had a program to insert profile information into a database. You track the following information about people:

First Name
Last Name
Favorite Fruit
Favorite Vegetable
Favorite Pasta
Favorite Meal
Rice or Beans
Favorite Desert
Favorite Dessert

Your application in use may look like this:

new-profile Bob Sacamano Apple GreenBeans Manicotti Steak Sahara Sundae

Your users will need to keep this whole list in their heads in order. That’s a pretty bad user experience and its likely that someone will forget a field or worse, get the order of the arguments wrong. This would add data into the wrong fields in your database. There are too many options to keep track of here, arguments aren’t a good choice for this program. Case-in-point: The above example forgets the “Rice or Beans” field. Did you catch it?

This application would be perfect for flags, so we’ll cover that in the next section.

Appendix and References

References

Advanced Bash-Scripting Guide: Exit Codes With Special Meanings
- There are few hard rules for exit codes. Exit 0 usually means everything worked as expected, no errors. Exit 1 usually means something went wrong. Exit 2 is commonly used to indicate a problem with the command itself (too many arguments, too few, etc). These “rules” and prefaced with “usually/commonly” because exit codes are ultimately up to the whims of the developer.

Software Engineering Essentials: Working with GitLab (and GitHub)

Wed, 13 Nov 2024 00:00:00 -0600

All writing for this post is licensed under CC-BY-4.0, while all code is licensed under the MIT license.

GitLab

Branch Etiquette

Git makes working in groups much easier than manually copying and pasting code together from multiple sources. A centralized Git hosting service can make it even easier. Most of the time when multiple people work on a project, changes are committed to feature-specific branches. These can be bug-fixes, new features, refactors, or anything else. The important thing to keep in mind is to keep your changes organized in the proper branches.

If you’re working on a bug-fix on a branch named front-page-css-bugfix, don’t pollute that branch with unnecessary changes like fixing test cases in an unrelated function. The changes in a branch should correspond to that branch. If you really want to do that work and save it somewhere, feel free to create a new branch. Branches are free and easy to use, feel free to make new ones as you need.

Merge Requests

Merge Requests (or Pull Requests in GitHub lingo) are a mechanism in most Git hosting services designed to make it easier to handle merging in new changes. While Git has a lot of tools for managing changes and merges, the web interface is a bit easier to get started with for newcomers.

More important than the change itself is the Code Review process that merge requests assist with. Chances are, your merge request won’t be accepted immediately and without question. Most of the time, we have something to fix or something to change. The merge request view allows other team members to comment on your diff and even highlight individual lines or blocks of changes to leave comments. If you’ve ever used the “Track Changes” feature of Microsoft Office product, you’ll be well-acquainted with this interface.

When a merge request is accepted, your commits in the merge request will be applied to the branch your request targets (usually the main development branch). Then, whenever people fetch and pull changes, your commits will be pulled down to their local repositories. This keeps everyone in lock-step with each other while you all work on the same codebase.

Your old branches should be deleted once merged. This isn’t a technical requirement, it just keeps things clean.

Protecting Branches

Because the main development branch is used for collaboration and keeping everyone on the project on the same page, you usually don’t want to commit changes directly to it. Instead, you commit changes to the main development branch through merge requests. This ensures that the commits people are adding are reviewed and communicated before they are added to the main branch.

Most Git hosting services have a feature that will allow you to protect certain branches. Protecting a branch can prevent things like deletion and/or directly pushing changes onto that branch. Instead changes are added in through merge requests.

Software Engineering Essentials: Version Control with Git

Mon, 11 Nov 2024 00:00:00 -0600

All writing for this post is licensed under CC-BY-4.0, while all code is licensed under the MIT license.

Git

What is Git?

“Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.”

Git is an application designed to manage your source code and make it easy to keep version history, but why would you want this? Programming gets messy, especially in large projects. Let’s say you’re testing an experimental bugfix, you may be tempted to copy/paste your code into a different folder and name it, my-project_bugfix, but what happens when you have my-project_bugfix2 or my-project_bugfix_real. Copying around folders of source code might work initially, but will quickly get messy, especially if you need to work with other people. Git provides a system to help keep your code organized, easy to work with, and easy to collaborate on.

A Git repository is essentially a place where you can store files to track and share changes to those files. Git repositories (or repos, for short) are usually used to store and track changes to source code files.

Installing Git

Git is available on a wide variety of platforms and you can find an installer here. There are some options to pick during setup:

Select Components: Default
Choosing the default editor used by Git: Use Visual Studio Code as Git’s default editor
- If you don’t have Visual Studio Code, exit the Git installer, install vscode, then re-launch the Git installer.
Adjusting the name of the initial branch in new repositories: Override the default branch name for new repositories: main
Adjusting your PATH environment: Git from the command line and also from 3rd-party software
Choosing the SSH executable: Use external OpenSSH
- SSH is now included in Windows 10 and Windows 11 by default
Choosing HTTPS transport backend: UUse the OpenSSL library
Configuring the line ending conversions: Checkout as-is, commit as-is
Configuring the terminal emulator to use with Git Bash: Use MinTTY (the default terminal of MSYS2)
Choose the default behavior of git pull: Only ever fast-forward
Choose a credential helper: Git Credential Manager
Configuring extra options: Check both Enable file system caching and Enable symbolic links
Configuring experimental options: Leave all options unchecked

Configuring SSH and Setting Up Your GitLab Account

First register for GitLab. Next, we need to create an SSH key so we can authenticate to GitLab.

Press Windows+R to bring up the “Run” dialog box.
Type powershell and press Enter to open up the PowerShell prompt.
Type ssh-keygen and press Enter to start the key creation process
- Press Enter to accept the default file path
- Type a passphrase to protect your key and press Enter.
  - NOTE: Your passphrase will not appear in the terminal window as you type it. This is a security feature.
- Re-type your passphrase and press Enter.
- The key is generated and a list of file paths is printed out. Make note of where the public key has been saved.
- Use VSCode to open that public key file. Keep powershell and vscode open for the next section.

Click here to go to your GitLab User Settings to manage your SSH keys. In the Add an SSH key section, copy and paste the entire public key file from before into the Key text box, then click Add key.

Now bring up the powershell window you had open before and use the ssh-add command. Type your key passphrase and press Enter. Remember: Your passphrase will not be shown here as you type it.

This command will add your key to an ssh-agent. This program just makes it easier to use your key. Instead of needing to type your passphrase every time you use the key, you just add it to the agent and use the passphrase only once. You’ll need to do this each time the computer reboots.

You can see what keys the agent has with the command: ssh-add -l

You can remove keys from the agent, thereby “re-locking” them, with the command: ssh-add -D

How Git views the World

A Brief Introduction to Hashes

Git’s view of the world is through hashes. A hash function is a way to “boil down” data to a fixed-length value. This sounds complex, but its simple in practice. Let’s look at the current hash of this file I’m writing now:

~ sha1sum Handouts/01-source-code-version-control-and-hosting/01-git.md
0b4597c1d9294a633ea584d715ce6681e57cb82a Handouts/01-source-code-version-control-and-hosting/01-git.md

But as I write more, change the content, and save, the hash will change:

~ sha1sum Handouts/01-source-code-version-control-and-hosting/01-git.md
8393d5018367212dcfe4fa6e477fdf5deff28576 Handouts/01-source-code-version-control-and-hosting/01-git.md

So hashing is a way to take data of any amount and get a fixed-length value that is unique to that data’s content. Some practical uses involve things like error checking, data deduplication, and protection against corruption or tampering. Git uses hashes to identify individual objects like files, directories, references, commits, and more.

Diffs on Diffs on Diffs

Git heavily utilizes the concept of diffs as a way to understand the differences between files (and multiple historical versions of the same file). This makes merges easier to manage and makes for more efficient file storage.

Let’s check out diff in action:

I have two files, a.txt and b.txt:

a.txt:

Test
This is file a

b.txt:

Test
This is file b

Here’s what I get when I run diff a.txt b.txt:

2c2
< This is file a
---
> This is file b

diff only shows us the differences between the files. Git can use this to figure out what changed between different objects. A commit in Git is like taking a snapshot of the diff of the entire repository. As you make more commits, you are creating historical records of what changed in that repository at that moment in time. By playing the diffs in reverse, you can effectively wind back the clock and travel to past, undoing changes.

So how does git “play back” those diffs? This is an operation called patching. A patch is a series of changes intended to be applied to an existing system.

So if I had a program and someone found a typo: hello.go

package main

import "fmt"

func main() {
	fmt.Println("henlo world")
}

They could send me a small patch to fix it:

typo-fix.patch

--- hello.go	2023-01-01 21:25:05.694853862 +0000
+++ hello.go	2023-01-01 21:26:15.863385402 +0000
@@ -3,5 +3,5 @@
 import "fmt"

 func main() {
-	fmt.Println("henlo world")
+	fmt.Println("hello world")
 }

And I could use patch hello.go typo-fix.patch or git apply typo-fix.patch to apply the diff. Now my application will print hello world!

A Git repository is just hashed objects and diffs stacked on top of diffs.

Extra Reading

If you enjoyed this section, feel free to check out Git Internals chapter in the official book for an ever deeper dive.

Using Git on the Command Line

Git is an open source application and there are a ton of products and graphical interfaces designed to make working with it easier. While these can work well in some straight-forward circumstances, they all tend to fall apart pretty badly when things go off the “happy path”, and it can be difficult to get back to a good state. For this reason, we’ll focus on using the command-line version of Git. Once you’re comfortable with this interface, you can absolutely use any Git-compatible application you want, including VS Code’s excellent Git integration, just don’t skip learning the CLI basics first.

Project Creation

A Git repository is just a folder with another .git folder inside of it. In the .git folder are a bunch of directories and files that Git uses to make sense of your codebase. These include hashed objects, diffs, and metadata used to keep things organized and human-compatible. You shouldn’t need to dive into this directory or change anything.

The command to instantiate a new Git repository on your local system is: git init my-project-name

Once you run this command, you’ll need a folder named my-project-name. All of your code, as well as anything you want to keep under version control (like documentation files), should go here.

Cloning an Existing Project

Git is a distributed system, which means projects don’t have to live purely on your local system. Many programmers utilize a Git-hosting service to store and share their code with other contributors. These include services like GitHub, GitLab, and Bitbucket. Don’t worry about signing up for any of these services just yet, the next section will walk you through the basics of GitLab.

Git hosting services give you an easy way to interact with other contributors, report and accept issues, manage merge requests, and most importantly: Share your code with the world. Its important to understand that Git is a distributed system by default, you don’t need these hosting sites to share or collaborate on code, but they do make it easier.

You can think about Git hosting services like YouTube or Vimeo. You can absolutely share video files with people or even make your own video hosting service, but using someone else’s service is so much easier. It is important to keep in mind that Git is not GitHub. GitHub can be used to host a Git repository, just like YouTube can be used to host a video. The hosting service and the core technology are two different things.

Most Git hosting sites will give you one or two Clone addresses you can use to copy down the project locally. Most often these addresses are SSH and/or HTTPS links. You can use these addresses in this command:

git clone <https://github.com/torvalds/linux.git>

or like this:

git clone git@github.com:torvalds/linux.git

Typically, an https repository link is going to be read-only. You can pull down changes, but won’t be able to push your own changes (unless you set up HTTPS authentication). An ssh repository link (the one that looks like git@gitlab) is typically going to be read+write. You will be able to push your changes to the remote repository.

This will clone the repository hosted on GitHub, by the user torvalds, with the project name linux, into a directory on your local system called linux. After you run that command, you can browse the Linux source code on your own system. Because Git is decentralized, you obtain a full copy of the source code repository, including history, commits, branches, tags, and other Git metadata. If you want, you can see what the code looked like last year, or even before then. You can see when certain features were merged in, or when a certain version-number was tagged.

One thing to keep in mind is some data found on the Git host site isn’t included with the repository. These would be things like merge request discussions, issues and bug reports, and project milestones and planning features. By default, Git does not track these items in the repository, so they are instead hosted directly by the Git hosting service.

Viewing and Committing Changes

Git’s version control isn’t automated, users need to manually make “checkpoints” of their changes to let Git know “these changes are important to track”. In Git, these “checkpoints” are called commits.

A commit is a way to tell Git “here are the changes I want to track”. Not everything in a repository folder is automatically tracked. We need to tell Git what files we want to track. For now though, let’s see how Git see’s the world:

In this example, I have a new repository and I’ve created my first file with some “Hello, World!” text in it. I can use git status to see what Git thinks of the files in the repository:

~/git/my-project-name git status
On branch main

No commits yet

Untracked files:
 (use "git add <file>..." to include in what will be committed)
 my-first-file.txt

nothing added to commit but untracked files present (use "git add" to track)>

Git sees a new file that it has never tracked before: my-first-file.txt

Now, let’s follow Git’s helpful advice, add the file, and re-run the status command to see what changes:

~/git/my-project-name git add my-first-file.txt
~/git/my-project-name git status
On branch main

No commits yet

Changes to be committed:
 (use "git rm --cached <file>..." to unstage)
 new file: my-first-file.txt

Now it looks like we have added the file to our staging area, but we haven’t committed. We’re only preparing the files for commit, we haven’t made the “checkpoint” yet. To do that, let’s use git commit. By default, a text editor will pop up or display in your terminal. This is used for writing a commit message. A commit message is what will be logged with your code changes into Git history. So if you’re working on a shared project, what you write here will be viewable by anyone with access to the repository.

Writing good commit messages is an art form and practice makes perfect. Try to include enough context with your messages, so a “code detective” down the line can figure out what your change did without needing to read the code itself. Keep in mind that you will be that “code detective” one day, and the code you’ll be digging into may very well be your own. Trying to track down a bug and running into a commit that just says “fix” is infuriating. Make it easy on yourself, write good commit messages.

A good Git commit message explains what changed and why. The first line is reserved for a title, the second line skipped, and the third line and beyond is for your longer write-up. This is so git log --oneline and other Git tools can display text nicely. This isn’t a hard rule though, people can (and will) disregard good commit message etiquette, the software will work with it all the same. Zach Holman has a great post about commit messages and why they just don’t matter, so there are multiple schools of thought on the subject.

Back to the example. So I’ve written this as my commit message:

Added my-first-file.txt to use in a Git example

I've added this file with default text in order to show Git examples
using commit messages.

I saved the file and exited the text editor. Git then uses this to construct a commit. It returned this text:

[main (root-commit) 3fa8cef] Added my-first-file.txt to use in a Git example
 1 file changed, 1 insertion(+)
 create mode 100644 my-first-file.txt

Now I can use git log to see my history:

commit 3fa8cefb1c854a550158a58d6443ec2b27e957fe (HEAD -> main)
Author: Tom Webster <tom@samurailink3.com>
Date: Wed Jan 4 00:17:48 2023 +0000

 Added my-first-file.txt to use in a Git example

 I've added this file with default text in order to show Git examples
 using commit messages.

To get a more compact view, you can use git log --oneline and that will display your commit titles in a list:

3fa8cef (HEAD -> main) Added my-first-file.txt to use in a Git example

You may have noticed that long string of jumbled text above: 3fa8cefb1c854a550158a58d6443ec2b27e957fe.

This is a commit ID. A commit ID is a hash used to identify the change you’ve added to history. Since hashes are designed to be unique, we only need to display enough characters to identify a single commit, we don’t need to display the full ID from above, we can just display 3fa8cef.

A simple example of this is like an ID system with too many digits. Let’s say you have a list of the best fruits, but each ID has 10 digits, so you end up with a table like this:

Fruit - ID Number

Apple - 0000000001
Orange - 0000000002
Banana - 0000000003
Strawberry - 0000000004

So much wasted space in that ID number. If we think we’ll have less than 100 fruits in this list, we can just use 2 digits to display the ID instead of the full 10:

Fruit - ID Number

Apple - 01
Orange - 02
Banana - 03
Strawberry - 04

Hashes in Git operate in mostly the same way: It will give you the shortest unambiguous hash, but the minimum number of digits is 4.

Git Remotes

Git Remotes are remote repositories. Because Git is a distributed system, you can have multiple remote repositories you can read from or write to. If you cloned this course repository from GitLab, you’ll probably see this when you run git remote -v:

origin <https://gitlab.com/samurailink3/go-twitter> (fetch)
origin <https://gitlab.com/samurailink3/go-twitter> (push)

By convention, the first remote added to a Git repository is named origin, but this is only a general convention, not a hard rule.

You can have multiple remotes and custom rules for pushing and pulling changes for each of them if you want. Most often you’ll just have one remote, hosted by one of the popular Git hosting services. This setup will allow you to push changes to an online repository and/or pull changes that have been pushed, but you haven’t pulled to your local repository.

If your repository was created on your local system only, without cloning from a Git hosting service, you won’t have any remotes configured initially. You can always add a remote with:

git remote add remote-name remote-address

Like this:

git remote add origin git@gitlab.com:samurailink3/go-twitter.git

Pulling Changes

Git has two different methods to pull changes into your local repository: fetch and pull. Fetching changes pulls down new objects, diffs, and references from a Git remote down to your local repository. Its important to keep keep in mind that fetching won’t change files in your local workspace. The changes are instead cached as part of that Git remote.

To fetch changes from the origin remote, you can run:

git fetch origin

If you’d like to fetch changes from all remotes, you can use:

git fetch --all

To actually update files in your local workspace, you would use git pull. This will import changes from the configured remote branch into your local branch. This means things like viewing history or checking diffs is near-instantaneous because all of the changes are cached locally, Git doesn’t need to go back to the internet to get more data. Once you fetch, those changes are stored locally.

Pushing Changes

Once you have commits you would like to push to your configured remote, you can use:

git push remote-name branch-name

Like this:

git push origin main

This tells Git, “Take my local commits and push them to the origin remote on the main branch. Git will automatically figure out which commits are missing on the remote and push only what’s needed to make them match.

Branches and Checkouts

Branches in Git are a way to create other “timelines” of history in your repository. So, if your main development branch is main, but you want to work on a complex feature without introducing bugs or breakages on the main branch, you can create a new branch named my-new-feature and start committing changes to that branch. By default, the first branch of a new Git repository is called main, or in older versions of Git, master.

To see a list of branches in your local repository, you can use:

git branch

To see all branches, even those present on remotes, you can use:

git branch --all

To create a new branch, you can just use:

git branch my-new-branch

Now, if you look at your list of branches with git branch, you’ll see it in your list of local branches:

* main
 my-new-branch

But if you look, you can see the * next to main, and if you run git status, you’ll see that you’re still on the main branch:

On branch main

So how do we switch branches?

git checkout my-new-branch

Now when you run git status, you’ll see:

On branch my-new-branch
nothing to commit, working tree clean

Now, when you commit changes, you’ll be committing to my-new-branch instead of main. To get back to the main branch, run git checkout main.

To save time, you can create and check out a new branch with one command:

git checkout -b my-new-branch

If you’d like to base your branch on a specific branch, you can use:

git checkout -b my-new-branch main

git checkout -b my-new-branch origin/some-other-branch

Additional Reading and Git Workflows

There are many strategies and theories on when you should make new branches and how you should organize them. When starting out: simpler is better. I’d recommend reading about GitHub Flow. The quick reference for this workflow is:

Create a new branch for your specific change
Make your commits
Push your branch to your remote
Create a merge request (called a Pull Request in GitHub’s terminology)
If you’re working in a group: Get sign off from your team and address any comments or requests
Merge your changes
Delete your branch

If you are working on software that needs multiple versions and maintenance releases (like operating systems or desktop software), something a bit more complex, like git-flow, could help.