Hi everyone, it has been a little while since I last wrote anything here. I didn’t stop training, though, which means the data have been piling up on my Garmin account. We will start doing some fun things with them very soon (next chapter, I promise).

The main reason for this short break is that I was running my R Fundamentals for Data Science workshop (more about that at the end).

One recurring piece of feedback I get from my students is how useful it is to see a complete workspace and script organised properly. How much clearer things become.

I hadn’t planned for it, but they convinced me I should also write about it here.

A reliable analysis needs a clean environment, and that environment has two components.

• There is the computer level: where you save your data and scripts, and how you separate inputs from outputs.
• Then there is the script level: how you organise the code itself.

All of this has only one purpose: making your work reproducible. By your colleagues or clients, but also by your future self.

These are only modest recommendations developed through my years analysing data. I am not pretending this is the best way of doing things. But this way works. Feel free to keep what makes sense to you and ignore the rest.

How to structure your files?

Define the headquarters of your analysis

Every analysis should have one main folder. I think of it as the project’s headquarters: everything the analysis needs or produces lives inside it.

In the context of this website, I consider each notebook an individual project. So the folder ~/DataSharp/enter-the-mind/running is my headquarters.

This folder will also become the working directory for R. Once that is set, every file can be located relative to the same starting point. Instead of pointing to a long, computer-specific path, we can simply refer to _inputs/DBs.

This makes the project much easier to move, share, and reopen six months later.

Imagine the stress this could remove from your shoulders if you weren’t scared to move things around on your computer.

Divide your headquarters into departments

Inside the headquarters, each folder should have one clear function. A simple project could start like this:

~/DataSharp/enter-the-mind/running/
   ├── _inputs/
   ├── _outputs/
   ├── _scripts/
   └── _temp/

• _inputs contains the original data and any other files needed by the analysis;
• _outputs contains the figures, tables, and datasets produced by your scripts;
• _scripts contains the code;
• _temp contains intermediate files that can be recreated and safely deleted.

The important part is not the exact folder names but the separation of roles. A raw input should never be confused with a processed output, and an intermediate file should never be mistaken for a final result.

An immediate benefit of this structure is that when you need to do some cleanup, you can simply delete everything from _outputs and _temp. You know everything in there has been generated by your scripts. Your input data are safe somewhere else and cannot be accidentally deleted.

I use an underscore in front of these names only to make sure the folders appear at the top of my list (my folders are always sorted alphabetically).

Give explicit names

When I open a project, I want to understand immediately how it is organised and where to look for what I need.

This may sound trivial, but give your files and folders explicit names (and later explicit variable names in R). Your future self will not remember what table_new2.csv contains. If you have updated files, keep the same base name but add the date. This way, you know they are different versions of the same file.

If scripts are meant to run sequentially, include their position in the name:

_scripts/
   ├── 01_import-data.R
   ├── 02_prepare-data.R
   └── 03_create-figures.R

Use a leading 0 to ensure that 02_prepare-data.R comes before 11_another-task.R.

Larger projects may need subfolders. For example, you may want separate output folders for figures and spreadsheets, or separate script folders if the list becomes too long. But only add these when they become useful. The structure should make the project easier to understand, not turn it into a maze.

How to structure your scripts?

The same logic applies inside a script. Each section should have one role, and the order should reflect the way the analysis is run.

Let’s extend the headquarters analogy. The project folder is your workshop: the place where the work happens. R packages are tools that you own, but you still need to bring them into the workshop before you can use them. The input data are the raw materials. The outputs are what you make from them.

How does that translate concretely into R?

Ready your tools

Start by loading all the packages needed by the script. Keeping them together makes the dependencies immediately visible.

library(dplyr)
library(fs)
library(ggplot2)
library(readr)

You do not need to know every package you will use before starting the analysis. If another one becomes useful later, scroll up and add it to this section rather than loading it in the middle of the script.

Locate your headquarters

Next, tell R where the project lives. This establishes the common starting point for every path used below.

setwd(
    path(path_home(), "DataSharp", "enter-the-mind", "running")
)

This is the only path that may need to change when the project moves somewhere else on your computer, or to another computer. Everything else remains relative to the headquarters.

I also like to create a few variables that describe the project and will be reused throughout the script. For instance, I create path variables pointing towards my input and output folders.

dir_inputs  <- path("_inputs")
dir_outputs <- path("_outputs")

This is also where I create the database connection object con we met in the previous chapter.

con <- DBI::dbConnect(              ## Open the connection
    RSQLite::SQLite(),              ## Specify the database type
    dbname = path(                  ## Point to the database file
        dir_inputs,                 ## Note how I am already reusing it
        "DBs",
        "garmin_activities.db"
    )
)

When you work with databases, you also need to close that database connection when you are done working with it. Where you write it may vary: either right after the last use of the database, or at the end of the script.

DBI::dbDisconnect(con)

Ready the raw material

Now you can easily locate and import the inputs.

my_data <- read_csv(path(dir_inputs, "my-data.csv"))

I also like to define important output paths here, so the script makes it clear where its results will be written.

In particular, you can create paths to all your different outputs here. This makes the script easy to update later if a file name needs to change. You do not need to create the complete list at the start. Think of it as a living list that grows with your project.

Because this example is written before I have real outputs, I’m using generic names, but please use explicit ones in your projects.

##- Output subfolders
dir_out_figures  <- path(dir_outputs, "figs")
dir_out_datasets <- path(dir_outputs, "data")
dir_create(c(dir_out_figures, dir_out_datasets))

##- Output files
f_updated_data <- path(dir_out_datasets, "cleaned-data.csv")
f_x_against_y  <- path(dir_out_figures, "x-y-scatterplot.png")

Remember that path() only describes where a folder should be; it does not create it. If the output subfolders do not exist yet, they are created with dir_create(). If they already exist, dir_create() will do nothing.

Similarly, the output files do not exist yet. They’ll be created later as your script develops. But their names and locations are defined here.

From this point onwards, the analysis can focus on the data rather than on finding files scattered around the computer. You can also move things around or rename folders without struggling to propagate these changes throughout the script.

How it looks at the end

Put everything together and you get a small template that can be adapted across projects:

# ---- Ready the tools ----
library(dplyr)
library(fs)
library(ggplot2)
library(readr)


# ---- Locate the headquarters ----
setwd(
    path(path_home(), "DataSharp", "enter-the-mind", "running")
)


# ---- Describe the project ----
dir_inputs  <- path("_inputs")
dir_outputs <- path("_outputs")

dir_out_figures  <- path(dir_outputs, "figs")
dir_out_datasets <- path(dir_outputs, "data")


# ---- Create structure iff needed ----
dir_create(c(dir_outputs, dir_out_figures, dir_out_datasets))


# ---- Name files that will be generated by the script ----
file_updated_data <- path(dir_out_datasets, "cleaned-data.csv")
file_x_against_y  <- path(dir_out_figures, "x-y-scatterplot.png")


# ---- Open database connection ----
con <- DBI::dbConnect(
    RSQLite::SQLite(),
    dbname = path(
        dir_inputs,
        "DBs",
        "garmin_activities.db"
    )
)


# ---- Ready the raw material ----
my_data <- read_csv(path(dir_inputs, "my-data.csv"))


# ---- Run the analysis ----

updated_data <-
    my_data |>
        slice_head(n = 10) ## Replace with your analysis

# Save your outputs as soon as they are generated to avoid saving
# a modified version later.
write_csv(
    updated_data,
    file_updated_data # The variable defined at the beginning
)



scatterplot <-
    ggplot(updated_data, aes(x = x, y = y)) +
        geom_point() ## Replace with your figure

# Save your outputs as soon as they are generated to avoid saving
# a modified version later.
ggsave(
    filename = file_x_against_y, # The variable defined at the beginning
    plot = scatterplot
)

# ---- Close open connections ----

DBI::dbDisconnect(con)

You do not need to write the complete setup before the analysis begins. Let it grow with the project. Just keep returning to the relevant section when you add a package, an input, or an output. That way, the script always reads from top to bottom and can be rerun from a fresh R session.

This may look like housekeeping—and, in a way, it is—but it is an essential part of the analysis. A clear structure reduces mistakes, makes collaboration easier, and gives you a much better chance of understanding your own work when you return to it later.

So I can only recommend adopting good habits from the start. And don’t trust the inner voice telling you that you’ll do it after you’ve made your script work. You won’t. There will always be something more urgent or important than that.

Do it proactively.

One final suggestion

Always start your analyses from a fresh R session and an empty environment. This is the only way to know that your script contains everything it needs. If it relies on an object left over from yesterday, it may work today but fail as soon as you or someone else tries to rerun it.

Old objects can also create particularly confusing bugs by silently replacing something your script was supposed to create.

So when R or RStudio asks, “Do you want to save your workspace?”, please say no. Always say no. Save the inputs and outputs you need deliberately, but do not preserve the entire state of your R session. There are far more reasons not to do it than to do it.

A brief word about the workshop

This short detour was inspired by my R Fundamentals for Data Science workshop. The workshop follows a complete analysis in R, from importing and inspecting data to transforming and visualising them, with reproducibility built into every step.

Project and script organisation are therefore taught alongside R, not as optional housekeeping at the end. If you are curious, you can read more about my workshop and my teaching approach.

Next chapter, we return to the running data. Promise kept.