How to Visualize and Plot Data

Let’s do some plotting!¶

Now that we have processed our data, let’s visualize the different input partners of our KCs.

Our goal is to answer:

For a given KC type, what pre-synaptic cell types provide input to it, and how strongly do they connect?

Before we open Plotly, let’s stop and think about what kind of plot could help us answer this question.

Take out a pencil and paper

Imagine that you were doing this analysis without a computer.

You have a list of different neuron types that provide input to a particular KC, along with the number of synapses they make onto that KC.

Take out a pencil and paper and sketch a way you might visualize this.

Think about something you have probably seen in school:

• We have different categories: the different input neuron types.

• We have a number associated with each category: the connection strength.

• We want to be able to compare the categories.

• We also want to order them from strongest to weakest.

What kind of plot might be useful for this?

Don’t worry about getting the “right” answer immediately. There are often several ways to visualize the same data. The important thing is to think about what biological insight you want before deciding which plotting function to use.

Now think about the axes

If you were to make a plot, what would you put on each axis?

We have two pieces of information:

1. Input neuron type

This is a categorical variable. For example:

PN1, PN2, PN3, PN4

2. Connection strength

This is a numerical variable. For example:

120, 85, 42, 15

So we need a visualization that allows us to compare a number across different categories.

Once you have an idea, try sketching it.

Then ask yourself:

How would I order the plot so that the strongest input partner appears first?

This is an important part of the visualization. We are not just interested in which neurons provide input, we also want to see which ones provide the strongest input.

Your turn! Before looking at the Plotly code, try to answer:

1. What type of plot would you use?

2. What would go on the x-axis?

3. What would go on the y-axis?

4. How would you order the input neuron types?

Once you have your idea, feel free to use your idea or you can follow mine below.

To me, a simple box plot is a good way to visualize this.

On the x-axis, we can have the input partner cell types, and on the y-axis, we can have the number of synapses.

Each plot will be dedicated to one KC cell type.

But why use a box plot rather than simply plotting one value for each input cell type?

Remember that so far, we have only removed duplicate neuron–neuron connections. We have not removed duplicate cell-type–cell-type connections.

For example, imagine that several PN neurons all provide input to the same KC type:

These are all connections between the same cell types, but they are different connections between individual neurons.

This gives us a population of neuron-level connections for each cell-type pair.

A box plot allows us to see the distribution of connection strengths within that population.

For each input cell type, we can therefore ask:

How strongly do individual neurons of this cell type tend to connect to neurons of this KC type?

The box plot can show us the median, spread, and range of these connection strengths rather than reducing everything to a single number.

We can then order the input cell types on the x-axis according to their typical connection strength, for example by their median.

This gives us a visualization where the strongest input cell types appear first, making it easier to compare the different partners.

So our visualization is answering:

For this KC type, which pre-synaptic cell types provide input, and how strong are those connections across individual neurons?

This is also a good example of why it is important to understand exactly what we have processed. We have collapsed connections across ROIs for individual neuron pairs, but we have deliberately kept the different neuron-to-neuron connections within each cell-type pair. That remaining variation is exactly what the box plot allows us to visualize.

Some advice on getting started with Plotly¶

Plotly Express and Plotly Graph Objects (go) are great visualization tools for exploring your data. One particularly useful feature is that Plotly creates interactive plots, which is especially handy when you are working with lots of cell types. You can zoom in, hover over points, and explore your data directly in the figure.

To get started, let’s say we want to make a box plot like the one above. A good first step is simply to search the Plotly documentation for the type of plot you want.

As you scroll through the documentation, you will find examples showing different ways to create and customize box plots. Try to understand what each part of the example is doing, and then adapt it to your own data.

For example, you might start with a basic box plot and then experiment with changing the axis labels, figure size, ordering, or other properties.

For more advanced customization, you can use Plotly Graph Objects (go). This gives you much finer control over individual parts of your visualization and opens up many more visualization techniques. We will cover this in a separate tutorial, but you can already explore the examples in the Plotly documentation if you are curious.

You may also have noticed that we used a categorical axis to order our cell types from the strongest input to the weakest.

A categorical axis is an axis containing discrete categories, such as cell types, rather than continuous numerical values. Plotly provides documentation on how to control the ordering and formatting of categorical axes.

Let’s interpret what we see!¶

We can see that the top five input cell types to KCa’b’-ap1 (once we zoom into the plot), ranked by number of synapses, are: `APL’, ‘VM1_lPN’, ‘DM1_lPN’, ‘DA1_lPN’, ‘VC1_lPN’

But what are these cell types? What do they do? And why might they be providing such strong input to this KC?

This is where we move from data analysis to biological interpretation.

A great resource for exploring Drosophila cell types is Virtual Fly Brain (VFB). VFB brings together cell-type annotations, anatomy, synonyms used in the literature, and references associated with those annotations. Virtual Fly Brain

Here is a tutorial on getting started with the VFB search. When searching, select Adult and use Type as your search criterion:

Virtual Fly Brain: Search tutorial

What should you look for?¶

Look for:

• What does the acronym stand for?

• What synonyms have been used for this cell type?

  • Particularly useful when searching the literature. A cell type may have been called something different in an older paper than the name used in our connectome dataset.

• What does the literature say about its function?

• Which papers are associated with the cell type?

Virtual Fly Brain: Understanding cell-type annotations

Your task¶

“What might the identity and known function of these input neurons tell us about the information being delivered to this KC?”

This is where connectomics becomes particularly interesting: the plot gives us an observation, and the literature helps us develop hypotheses about what that observation might mean.

Other ways of visualizing to gain insight¶

Do different KC cell types receive inputs from similar cell types and strengths?

Remember that so far we have looked at one KC type, KCa’b’-ap1, at a time. A heatmap gives us a way to compare the input patterns of many KC types simultaneously.

A heatmap is essentially a grid where the rows and columns represent categories, and the value in each square is represented by its color.

For our question, we could have:

• Rows: KC cell types

• Columns: input cell types

• Color: connection strength (med_type_pre_sum_weigh)

First, we need to change the shape of our data:

Our dataframe is currently in long format. That means we have many rows, with each row representing a connection:

type_pre       type_post        med_type_pre_sum_weigh
APL            KCa'b'-ap1      120
VC1_lPN        KCa'b'-ap1       85
VA6_adPN       KCa'b'-ap1       72
APL            KCa'b'-ap2       95
VC1_lPN        KCa'b'-ap2       20
...

This is useful for many analyses, but a heatmap naturally works with a grid.

We therefore want to reshape our dataframe into wide format:

             APL   VC1_lPN   VA6_adPN   VA3_adPN
KCa'b'-ap1   120      85         72         64
KCa'b'-ap2    95      20         61         10
KCγ           12      78         15         70

Here, each row is a KC type, each column is an input cell type, and each value represents the strength of that connection.

How do we make this?

We want:

type_post → rows

type_pre → columns

sum_weight → values

What pandas function might allow us to rearrange our dataframe this way? If you are not sure, this is another good opportunity to search the pandas documentation or ask an AI model. Once you have figured that out, we can use the resulting dataframe as the input to our heatmap.

For example, pandas’ pivot() function can perform this reshaping.

But first, let’s prepare our data

Remember that we need to go back to our original input_conn_df_v2, which contains all of our KC types. We previously used a subset of this dataframe to investigate a single KC type, but now we want to compare all of them.

First, we calculate the median connection strength for each cell-type pair.

There can be many neuron-to-neuron connections between the same two cell types, so we group by type_post and type_pre and calculate the median sum_weight. This gives every neuron-to-neuron connection belonging to the same cell-type pair the same median value.

We only need one value for each cell-type pair for our heatmap, so we can now remove the duplicate cell-type pairs. Notice that we are removing duplicates based on cell type, rather than bodyId_pre and bodyId_post.

Earlier, we removed neuron–neuron duplicates (after summing the number of synapses) because the same pair could appear across multiple ROIs. Here, we are doing something different: we have already calculated our median across the neuron-level connections, and now we only need one row representing each cell-type → cell-type relationship.

We can then also remove the columns that we no longer need for this visualization.

Now let’s make our dataframe wide¶

We can use pivot():

Think back to our three questions:

What goes on the rows? → type_post

What goes on the columns? → type_pre

What goes inside each cell? → med_type_pre_sum_weight

Finally, some KC–input cell-type combinations may not exist in our dataset. These appear as NaN after pivoting.

For this visualization, we want a missing connection to be represented as zero, so we can replace the NaN values:

Now heatmap_df has the structure we need to make our heatmap.

The important thing to notice is that we have transformed our data several times because the structure we need depends on the question we are asking.

This is a common pattern in data analysis: the same biological data can be represented in different ways to answer different questions.

Now let’s finally make a heatmap using plotly!

Clustering can help us find patterns¶

You may notice that there are some similarities in the heatmap, but they can be difficult to spot when we have many KC and input cell types.

This is where clustering can be helpful.

Instead of looking at every row and column individually, we can ask:

Which input cell types tend to connect to similar groups of KCs?

Clustering groups rows or columns that have similar patterns in the data. In our case, this means that KC types with similar input profiles can be placed close together, while KC types with very different input profiles are placed farther apart.

For example, imagine two KC types:

          APL   PN1   PN2   PN3
KC1        20    15     2     1
KC2        18    17     3     2
KC3         1     2    20    15

KC1 and KC2 have very similar input patterns, while KC3 receives a very different combination of inputs.

A clustering algorithm could therefore place:

KC1
KC2
       together

KC3
       separately

This can reveal structure that is difficult to see by simply looking at the heatmap.

How does the clustering actually work?¶

There are many different ways to cluster data. Here, we will use hierarchical clustering based on correlation.

The basic idea is simple:

If two KCs have similar patterns of inputs, they should be considered similar.

We can use correlation to measure how similar their input patterns are.

For example:

KC1 = [20, 15, 2, 1]
KC2 = [18, 14, 3, 2]

These two KCs have a very similar pattern: they receive relatively strong input from APL and PN1, and relatively weak input from PN2 and PN3.

Their correlation would therefore be high.

In contrast:

KC3 = [1, 2, 20, 15]

has almost the opposite pattern, so its correlation with KC1 and KC2 would be much lower.

We can convert correlation into a distance:

High correlation → small distance → more similar

Low correlation → large distance → less similar

One common way to do this is:

distance = 1 - correlation

So a correlation of 1 gives a distance of 0, meaning the two profiles are identical in their pattern.

Why is it called hierarchical clustering?¶

We start by treating every KC as its own group:

KC1    KC2    KC3    KC4    KC5

We then find the most similar KCs and join them together.

For example:

KC1 ──┐
      ├── Group 1
KC2 ──┘

KC3 ──┐
      ├── Group 2
KC4 ──┘

KC5

We then continue combining the groups based on their similarity until eventually all KCs belong to one large hierarchy.

This produces a dendrogram:

             ┌── KC1
         ┌───┤
         │   └── KC2
     ┌───┤
     │   └──── KC5
─────┤
     │   ┌──── KC3
     └───┤
         └──── KC4

The dendrogram gives us a visual representation of the relationships between the KC input profiles.

We can then reorder our heatmap according to this hierarchy so that KCs with similar input patterns appear next to one another.

One important caveat: the resulting clusters are mathematical groupings, not automatically biological categories. A cluster tells us that cell types have similar connectivity patterns according to the metric and clustering method we chose. It is then up to us to investigate whether those patterns have a meaningful biological explanation.

Here is a great video on hierarchical clustering: https://www.youtube.com/watch?v=7xHsRkOdVwo

Here we can see that APL provides input to all KC types. In contrast, some KC types—such as KCg-m, KCa’b’-ap2, KCa’b-s, KCa’b-c, and KCa’b-m—appear to share more similar input profiles.

You can now explore these groups as an exercise: What are the known roles of these KC types? Can you find evidence in the literature that might help explain why they receive similar or different inputs?

One interesting observation from this analysis is that not all Kenyon cells receive the same inputs. Different KC types appear to have distinct patterns of connectivity, which may help explain why they are classified as different cell types in the first place.

Remember, however, that seeing similar connectivity patterns does not by itself tell us why these cells are different. It gives us a hypothesis to investigate.

Normalization¶

So far, we have been using the raw number of synapses as our measure of connection strength.

But let’s think about what that means biologically.

Imagine two KC types:

Both KCs receive 20 synapses from Partner 1. If we only look at the raw number of synapses, we would say that Partner 1 has the same connection strength to both KCs.

However, these KCs receive very different amounts of total input from the partners they connect to. KC-1 receives 1,000 incoming synapses, whereas KC-2 receives only 100.

So is a connection of 20 synapses really equivalent for both KCs?

For KC-1:

Only 2% of its incoming synapses come from Partner 1.

For KC-2:

Now 20% of its incoming synapses come from Partner 1.

So although the two connections have the same raw strength, Partner 1 accounts for a much larger proportion of KC-2’s input.

This is where normalization becomes useful.

Instead of asking:

How many synapses does Partner 1 make onto this KC?

we can ask:

What proportion of this KC’s total input comes from Partner 1?

We can calculate this as:

For our example:

KC-1: 20 / 1000 = 0.02 → 2%

KC-2: 20 / 100  = 0.20 → 20%

Now we can see something that was hidden when we only looked at raw synapse counts: Partner 1 contributes much more strongly, proportionally, to KC-2 than to KC-1.

This is an important consideration when analyzing connectomic data. There is no single “correct” measure of connection strength—the best measure depends on the biological question we are asking.

Raw synapse counts tell us about absolute connectivity, whereas normalized values can tell us about the relative contribution of a partner to a cell type’s overall input.

The tutorial ends here for now!¶

At this point, you have explored several ways of asking questions about connectomic data—from identifying input partners, to comparing connection strengths, to clustering KC types based on their connectivity patterns.

But the analysis does not have to end here.

Think about how you could incorporate the normalized connection strengths we just discussed into the analyses you have already performed. How might the patterns change if you consider the proportion of a KC’s total input rather than the raw number of synapses?

More importantly, think about the biological question you are trying to answer. Different questions may require different ways of processing and visualizing the same data.

Connectomics can be used to ask many different questions, and answering one question often leads to several more. There is not always one “right” way to analyze the data. What matters is that you:

• Clearly define the biological question you are trying to answer.

• Make sure your code does what you think it does.

• Understand the assumptions behind your processing choices.

• Think critically about the limitations of your analysis.

• Consider whether your results actually support the biological conclusions you are drawing.

As you explore the fly brain, try not to think of coding as simply a way to get an answer. Instead, think of it as a tool for turning biological questions into testable analyses.

Have fun exploring the fly brain!