EffConPy: Open Source Causality Discovery in Python

Let me tell you my experience

🔎 What is Causality?

Traditional statistical and machine learning methods mostly focus on correlations, but causal models allow researchers to infer mechanisms and predict the effects of interventions.

Nevertheless, the discovery of cause-and-effect relationships has been always attractive.

Indeed, despite being used in many types of data, this is particularly relevant for time series data, for example, the stock market or clinical data as the cerebral activity. Non-time series cases can be based on causal networks of discrete events where one event causes another or more:

Flu → Fever

↘ → Cough

Rain → Slippery Road → People falling

A user Likes Sci-Fi → Recommend him to buy or watch "Interstellar"

Those might appear trivial chains, but they can be expanded to many components (also called larger networks) providing insightful inputs that can only be reached computationally given the large amount of data. With the rise of machine learning and deep learning, the need for robust causal inference tools is growing even more compared to the past.

Most models from the nearly century-old autoregressors or the more recent zero-shot large foundation models (like TimesFM, CHRONOS, LLMTime)—focus on forecasting values. But they often leave one burning question unanswered: "Does one time series actually influence another?"

For example, consider the surge in demand for Nvidia cards. A time series prediction could be able to predict rising stock prices for Nvidia, but what about the ripple effects on semiconductor supplier shares like TSMC? And what if the changes there will be even more pronounced and therefore more relevant (even if caused by another stock share)?

That's where causality-driven AI comes into play.

🛠️From Correlation to Causation

Emerging from econometrics, the most common approach is the Granger causality. This was introduced by the British economist Clive Granger more than 5 decades ago, and it has been the go-to technique since it has low computation and is therefore easily scalable. It’s simple: if changes in series X help predict series Y after some time, then X is said to “Granger-cause” Y.

Here’s a quick Python snippet using the Statsmodels library:

import statsmodels.api as sm
from statsmodels.tsa.stattools import grangercausalitytests

# Load sample macroeconomic data
data = sm.datasets.macrodata.load_pandas().data[["realgdp", "realcons"]].pct_change().dropna()
gc_results = grangercausalitytests(data, maxlag=4)

The “lag” in the last row is used to indicate how back/far in time we should look for relationships (in brain activity from functional magnetic resonance imaging is generally “1”)

While handy, Granger causality is inherently linear. It struggles to capture the non-linear interplay that is so common in complex systems. Obviously, if we are interested in just 2 time series, it is enough. «Complex» means many parts.🚑

🌟 The Stuff Inside EffConPy:

EffConPy addresses the aforementioned challenges; it is easily installed by using the command «pip install effconnpy», and it is developed by a team of people led by Dr. Alessandro Crimi from the technical university of Krakow. The library includes multivariate Granger causality, which even if based on Statsmodel, is not out-of-the-box (thank you for putting it together), and more modern approaches such as dynamic Bayesian networks, convergent cross-mapping, etc., which might detect subtle, non-linear-causal-influences-that-traditional models might miss. They are all put together, and therefore, it allows straightforward benchmarks, just changing the method in the last row of this snip, for example:

from effconnpy import CausalityAnalyzer
import numpy as np
# Generate sample time series
data = np.random.rand(100, 3)
analyzer = CausalityAnalyzer(data)
results = analyzer.causality_test(method='ccm')

If you are wondering why we need something more than Granger causality, let me explain it to you here: Granger causality is explicitly temporal (although some people complain that we have the so-called temporal correlation). It checks whether past values of one variable improve the prediction of another beyond its own past values. However, it assumes linear relationships and does not capture causality in complex dynamical systems where interactions are nonlinear or bidirectional.

Dynamic system approaches like Convergent Cross Mapping (CCM) and Transfer Entropy (TE) are focused on those aspects. CCM, based on state-space reconstruction from chaos theory, detects causality by checking if one variable's historical states can reconstruct another’s, implying a shared underlying dynamical system. The math related to this is not just a regression in time. It is based on advanced concepts from the dynamical system of the butterfly effect of the Lorentz attractor:

This is nothing esoteric, it is just math particularly useful for feedback systems and ecological interactions. TE, rooted in information theory, quantifies how much knowing one variable reduces uncertainty in another, making it well-suited for detecting nonlinear and asymmetric dependencies. Until now, I have not seen a Python framework putting all this together.

🕵️What I Did NOT Find Inside Effconnpy

There has been a growing interest in large foundation models (LFM) for time series prediction, such as TimesFM. Those models are good predictors and they do little work with causality. Therefore, it is understandable that they have been left out by a causal toolbox. However, they are probably the future. Indeed, we need little work to turn them from predictor to causality. Like with Granger, we could check if one series helps predict another one.

The other missing thing is reservoir computing causality. A reservoir is not something from your pipe. It is a dynamic, randomly connected network (metaphorically like fluid water), which, in response, to input data, generates intricate patterns. In practice, it is designed by recurrent neural networks. So, after all, it is AI.

The patterns created within the «reservoir» help us reveal deeper causal links, such as how fluctuations in one stock market might ripple through related industries or how a stroke disrupts neural connectivity in the brain. This approach was recently introduced in an IEEE Access study titled “End-to-End Stroke Imaging Analysis Using Effective Connectivity and Interpretable Artificial Intelligence” by Wojciech Ciezobka and colleagues, who applied these advanced techniques to healthcare. They used effective connectivity analysis to examine how the activity in one brain region influences another—vital for understanding stroke impacts. It will be great if also this will be included.

📝 Conclusion

Causality studies have always been attractive for people interested in time series analysis. By studying the temporal relationships between elements, especially their cause-effect, we allow us to understand better the intricate dynamics that govern our world from the available data, independently if we are talking of economic indicators or the connectivity within the human brain.

If you want to know more? As I said, it is open-source and therefore available on the related GitHub repository, or keep the conversation on how causality-driven AI is redefining our understanding of time series data, commenting here.🚀🧠