Before we begin

If you’ve found yourself here, we probably have something in common.

Perhaps you’ve realised that there is more to data science than writing code or chasing whatever tool happens to be fashionable this week.

Perhaps you’ve also grown a little tired of the constant buzz around AI.

What I often miss when I read about data science are the decisions behind the analysis. Where were the data found? How were they curated? Why were they transformed this way?

Why is that? The process of solving problems is not always a pretty story but it’s also the most exciting part. So why are the most interesting parts of the journey so rarely discussed?

I don’t know the answer. But I suspect I’m not the only person who would rather read about how data scientists actually get their hands dirty than about how the latest AI model has made everything (allegedly) easier, faster, better.

I hope you do, too.

Why these notebooks

I believe most complex data problems can be solved through a succession of simpler steps. Complexity is greatly reduced once you clearly define where you want to go and have the discipline to continue in that direction.

The challenge is to find that path of least resistance for you.

And for me, you must have clarity on two elements:

1. where you want to go with your project, and
2. where you currently are, which means you must have a clear idea of what your data truly are and what can be objectively answered from them.

That’s how I’ve always worked, whether I was researching past climate change, freelancing on data science projects, or preparing workshops to teach data science.

Different activities, one process: clarify the question, gather and curate data, and assess whether the latter can inform the former. Only then can I determine that path between A and B and break it into more manageable parts.

These notebooks are where I allow myself to think out loud. Where I document my thought process. Where I share with you the nuggets of knowledge I’ve collected along the way.

What you can expect

Some notebooks will take us somewhere interesting. Others may not. That’s part of the process.

My end goal is not to share flashy new findings about whatever topic I investigate. I want to show you how I got there.

The subjects are simply the excuse. The process is what I hope you’ll take away.

My goal is to document a way of thinking, a method, that you can borrow, adapt, and eventually make your own.

I hope you’ll enjoy following some of my investigations, and perhaps even start one of your own.

Thank you for stopping by.

— Manuel