Turning a Biological Questions into Code

Turning a biological question into code¶

When figuring out how to answer a biological question using code, it can help to break the problem down into two steps:

1. What data do I need to answer my question?

   In this case, we want to identify the neurons that send input to different KC types. We can find these using the type_pre column, which tells us the input neuron type, and the type_post column, which tells us the KC type receiving that input.

2. How can I process and visualize the information I need?

   There are many ways we could approach this. For now, we want to answer two simple questions:

   What different neuron types provide input to each KC type?

   Can we order these input partners based on how strongly they connect to each KC type?

Before we can answer these questions, however, we need to process our data.

For example, suppose we simply counted how many times each input partner appears for a given KC. We would get the wrong answer!

Why? Remember that our data are organized by synaptic connections within ROIs. The same pair of neurons can therefore appear multiple times in our dataframe if they form synapses in different brain regions.

For example:

These rows all describe connections between the same two neurons. To find the total number of synapses between these neurons, we need to combine these rows and add their weight values:

5 + 8 + 3 = 16 synapses

So before we look at KC input partners, we first need to combine all rows belonging to the same pair of neurons and sum their synaptic weights.

Notice how we spot the same bodyId_pre bodyId_post connection occuring because the synapses are in different ROIs?

Using AI responsibly¶

A useful pandas function for this task is .groupby(). But if you are not sure which function you need, this is a good opportunity to search the internet or ask a generative AI model.

However, there is an important distinction between using AI to help you learn and using AI to do the thinking for you.

If you simply copy and paste the code an AI gives you without understanding it, you may get your immediate problem solved, but you are missing an opportunity to develop your own programming skills. AI can also make mistakes, even when its answer looks convincing.

Instead, try explaining your problem to the AI and asking it to help you reason through it.

For example, you could ask:

I have a dataframe with columns bodyId_pre, bodyId_post, weight, type_pre, type_post, instance_pre, and instance_post.

bodyId_pre and bodyId_post identify a synaptic connection between two neurons, and weight represents the number of synapses between them within a particular ROI. The same pair of neurons can therefore appear in multiple rows if they connect in different ROIs.

I want to combine all rows belonging to the same bodyId_pre and bodyId_post pair and add their weight values. What pandas functions might help me do this?

Notice that we are not simply asking:

“Give me the code.”

Instead, we have described the biological structure of our data and the problem we are trying to solve. This forces us to think about what the data represent and what kind of operation we need.

If this is still difficult, that’s okay. You can have a conversation with the AI and ask it to help you think through the problem step by step. You can even say:

“Don’t give me the answer yet. Help me think through what I need to do.”

The goal is to use AI as a learning tool, rather than as a replacement for your own reasoning.

Go to the documentation — always!¶

After thinking through the problem and asking AI for guidance, you may learn that .groupby() and .sum() are useful functions for this task.

Now go to the documentation and look them up:

• DataFrame.groupby()

• DataFrame.sum()

Don’t just copy the example from the documentation. Try to understand what each function is doing.

Look at the examples. Try changing them. See what happens when you group by different columns or change what you are summing.

Then apply what you have learned to our connectome data.

There is no single “correct” way to process connectomics data¶

This is one of the challenging parts of working with synapse-level connectomics: there are not always hard-and-fast rules for how data should be processed.

Connectomics is a relatively new field, and different biological questions can require different processing decisions. For example, whether we sum synapses across ROIs, how we define a connection, or which connections we include can all affect the results we obtain.

This means that an important part of your learning is to think carefully about the caveats and assumptions behind your processing decisions.

Whenever you make a processing decision, take note of what you did and why you did it. This will become especially important as your analyses become more complicated.

When I was learning, I would heavily comment my code, not just to explain what the code was doing, but also to remind myself why I had made a particular decision. This might be helpful for you too.

And don’t feel that you have to figure everything out on your own. Come to office hours and ask questions, email us when you are unsure, or use generative AI as a tool to think through a problem.

Getting stuck, questioning your assumptions, and figuring out why something works (or doesn’t) are all part of learning connectomics!

Here, we are grouping the dataframe by bodyId_pre and bodyId_post.

This means:

“Put together all rows that represent the same pair of neurons.”

Then, .transform('sum') adds the weight values for each group.

For example, if we have:

The new summed_weights column will contain 16 for all three rows:

Why does transform() repeat the value?

Because transform() returns a result with the same number of rows as the original dataframe. This allows us to add the result back as a new column.

Why not just use .sum()?

You might be wondering why we are using .transform('sum') instead of simply using .sum().

Try:

input_conn_df.groupby(
    ['bodyId_pre', 'bodyId_post']
)['weight'].sum()

.sum() also adds the weights together, but it collapses each group into a single result.

For our example, we would get:

This can be useful if we want to immediately create a new dataframe containing one row per neuron pair.

However, here we are taking a slightly different approach. We first want to calculate the total connection strength and add it to our existing dataframe as a new column. That is why we use:

.transform('sum')

This gives us the total while keeping the original rows (we want to keep other information like cell types!), allowing us to then remove the duplicates ourselves.

So, as a rule of thumb:

• .sum() → collapse each group into one result

• .transform('sum') → calculate the group sum while keeping the original rows

Remove the duplicate rows

We now know that all three rows represent the same connection between neurons 1001 and 2001. We don’t need to keep all three rows anymore.

input_conn_df_v2 = input_conn_df.drop_duplicates(
    subset=['bodyId_pre', 'bodyId_post'],
    keep='first'
)

Here, drop_duplicates() looks specifically at the combination of bodyId_pre and bodyId_post.

It keeps the first row for each unique neuron pair and removes the other rows.

Our example therefore becomes:

We don’t actually care which ROI is shown anymore, because we have already combined the synapses across ROIs. The important information is now the neuron pair and their total number of synapses.

Why do we need both steps?

It is important to understand that transform('sum') and drop_duplicates() are doing different jobs.

transform('sum') answers:

How many total synapses connect these two neurons?

drop_duplicates() answers:

How can I represent this neuron pair only once in my dataframe?

Together, they turn our ROI-level data into a dataframe where each pair of neurons appears only once, with its total connection strength recorded in summed_weights.

This is important because if we kept the duplicate rows, we could accidentally count the same biological connection multiple times in our later analysis.

Synapse Threshold¶

It is common in connectomics to apply a synapse threshold when defining connections. Synapses can be missed during reconstruction, but false-positive synapses can also occur. As a result, very weak connections may represent spurious or uncertain connections.

In this tutorial, we will use a threshold of 5 synapses: connections with fewer than 5 synapses will be removed from our analysis. Most analysis follow this approach. Importantly, the appropriate threshold depends on the specific analysis and dataset being used.

Note: The threshold of 5 synapses is a choice for this tutorial, rather than a universal rule. Different analyses may require different thresholds.