Traditional ML in practice: predicting follow-on funding from Crunchbase data

In traditional machine learning you build the model yourself. You choose the data, define the label, engineer the features and decide how it is evaluated. Every decision is yours, and so is every finding.

This project answers a venture-capital question: after a funding round, will the company raise another one within two years? The final model ranks rounds well enough that the top tenth raise again at 69%, against 42% on average, and it shows which signals drive that.

Below, the project runs in the order an ML engineer would run it: the goal first, then twelve implementation steps, then the results against each objective.

1.Goal, objectives and planning

The business question. An early-stage investor backs dozens of companies and cannot support all of them equally. Knowing which portfolio companies are likely to raise a follow-on round helps decide where to spend partner time, when to set aside reserves for follow-on investments, and which companies may need help before they run out of money. It is the venture-capital case study from The art of AI engineering, made concrete.

Objectives. Three, each with a success criterion fixed before training:

Data. The public Crunchbase snapshot published on Kaggle as “Startup Investments”[1], a relational dump that ends in December 2013. Three of its tables matter: funding_rounds (one row per round: date, type, amount, participants), investments (one row per investor in a round) and objects (companies, with category, country and founding date).

Label. target = 1 if the same company raised another round within 24 months of this one.

Evaluation plan. Train on earlier years, test on later years, compare against baselines, and report ROC-AUC for ranking quality, plus PR-AUC and top-decile precision for shortlist quality.

Stack. Python 3.12 with pandas and NumPy, scikit-learn for the models, matplotlib and Jupyter for analysis. Everything runs on a laptop CPU.

2.Implementation, step by step

Twelve steps in three phases. Three of them decide whether the score can be trusted at all: the label, the features and the split. They get the most attention below.

Step 1. Data collection

The unit of prediction is a funding round, so the modelling table starts from funding_rounds: 52,928 rounds from 31,719 companies. Company attributes come from objects. Investors come from investments, aggregated to one value per round, because a round with five investors appears there five times.

Step 2. Exploratory data analysis (EDA)

• Duplication risk. Rounds with recorded investors average 2.4 investor rows each, up to 36. Joining from the investments table would copy each round that many times.
• Missing values. Valuations are almost never recorded, more than a third of rounds list no participants, and founding dates are missing for 18% of rounds.
• Skew. The median disclosed round is $2.3 million and the largest $3.8 billion, and 11% of rounds report $0, which means “not disclosed”, not “nothing raised”.
• Time. The data grows from 1,633 rounds in 2005 to 11,072 in 2013, and stops in December 2013.
• An unreliable flag. Crunchbase’s own is_last_round marks 6,481 rounds as “last” even though the same company has a later round in the data, so it can’t serve as the label.

The time pattern matters most. A round from late 2013 had no time to be followed by another, so it looks like a dead end whatever happens next:

Step 3. Data cleaning

• Dropped 248 rounds with no date and 19 with implausible dates before 1990.
• Kept one row per funding_round_id.
• Treated $0 amounts as missing, with an amount_missing flag, so the model doesn’t read an undisclosed round as a tiny one.

Step 4. Label definition

The label is a product decision: it turns the business question into something a model can learn. “Raises another round” needs a time limit, or rounds that are followed up years later count the same as rounds followed up in months, and every recent round looks like a dead end.

• Fixed horizon: target = 1 if the next round came within 24 months.
• Censoring: rounds after December 2011 are dropped, because they don’t have 24 months of future in the data.

That leaves 31,252 labelled rounds, 38% of them followed by another round.

rounds = rounds.sort_values(['object_id', 'funded_at'])
rounds['next_round_at'] = rounds.groupby('object_id')['funded_at'].shift(-1)
horizon = pd.DateOffset(months=24)
rounds['target'] = (rounds['next_round_at'] <= rounds['funded_at'] + horizon).astype(int)
rounds = rounds[rounds['funded_at'] <= rounds['funded_at'].max() - horizon]   # censoring

Step 5. Feature engineering

One rule for every feature: use only what was known on the round date. These are called point-in-time features.

g = rounds.groupby('object_id')
rounds['rounds_so_far']     = g.cumcount()
rounds['raised_so_far']     = g['raised_usd'].cumsum() - rounds['raised_usd']
rounds['months_since_prev'] = (rounds['funded_at'] - g['funded_at'].shift(1)).dt.days / 30.44
rounds['log_raised']        = np.log1p(rounds['raised_usd'])   # log(1 + x) keeps $0 at 0

Money goes on a log scale and dates become ages and gaps. The tempting shortcut is the company totals in objects, such as funding_rounds and funding_total_usd. They are measured at the snapshot, so they include the rounds after the one being predicted: if a company’s total is one round, that round is by definition the last. A model given them learns Crunchbase’s bookkeeping, not investor behaviour.

Step 6. Feature selection

Selection here is a leakage screen first and a statistics exercise second. Anything measured after the round date goes: snapshot totals, the company’s eventual status, and milestone and relationship counts. What remains are ten point-in-time features (those above plus two flags, for a recorded valuation and a known investor) and the encoded categories, few enough that automatic selection such as SelectKBest adds nothing.

To confirm the screen, I added the two snapshot totals back as a test. ROC-AUC jumped from 0.69 to 0.965, far too good for a question investors are paid to answer: the columns contain the answer, so they stay out.

Step 7. Train/test split

A random split by row is the default, and here it would be wrong. Rounds of the same company share company-level features, so a random split lets the model recognise the company instead of judging the round.

The split is by time instead. Training uses rounds from 2005–2009 (15,273 rounds, 35% followed up) and testing uses 2010–2011 (15,058 rounds, 42%). This is how the model would be used: trained on the past, applied to the next rounds. All preprocessing (imputation, scaling and the category vocabulary) is fitted on the training years only.

Step 8. Baselines and model training

Baselines first, so every model has to earn its place:

1. Majority class: always predict “no follow-on”.
2. A one-line rule: score each round by how many rounds the company had already raised.
3. Logistic regression: linear and interpretable.
4. Random forest: 300 trees, at least 20 rows per leaf.
5. Gradient boosting (HistGradientBoostingClassifier): handles missing values natively.

Step 9. Hyperparameter tuning

Gradient boosting was tuned over 18 settings (learning rate, leaf size, number of trees) on a validation slice inside the training years, mid-2008 to 2009. The best setting scored 0.689 on the test years, against 0.690 with the defaults. Tuning was not where the value was; the data, the label and the split were.

Step 10. Evaluation

Four measures, each answering a different question: ROC-AUC for how well the model ranks rounds, PR-AUC because positives are the minority, top-decile precision for the shortlist use case, and the Brier score for how well-calibrated the probabilities are.

Step 11. Model interpretation

Permutation importance on the test years: shuffle one feature at a time and measure how much ROC-AUC drops. It measures what the model depends on in data it has never seen.

Step 12. Deployment

To serve the model, save the whole scikit-learn pipeline, encoding and imputation included, so an API can take raw values such as "series-a". Saving only the model and re-creating the encoding elsewhere invites training-serving skew: inputs in production that don’t match what the model was trained on. The API itself is the next step and is not built yet.

3.Results

Objective 1: better than simple rules

Met. The gain over a sensible rule is real (0.61 to 0.69) but modest, and the forest and boosting are almost tied. Most of the signal is simple and close to linear; the trees add interactions on top.

Objective 2: a useful shortlist

Met. Ranked by the model, the top 10% of 2010–2011 rounds raised again within 24 months at 69%, against a 42% average: 1.6 times the base rate. The bottom 10% raised again at 13%. For a portfolio review, that separates “likely to raise, set aside reserves” from “unlikely to raise, look closer” well enough to act on.

Objective 3: the drivers

Met. Four signals stand out, and an investor would recognise each of them:

• Stage. 35% of first rounds are followed up within two years, against 62% from the fourth round on.
• Momentum. Companies that raised less than six months earlier raise again 61% of the time, against 40% after a gap of more than two years.
• Investor track record. Rounds led by an investor with 20 or more earlier deals are followed up 50% of the time, against 36–40% for everyone else.
• Age and country. These lead the permutation importance. Companies aged two to eight years raise again most often (50%), and a missing founding date is a warning sign (28%): thinly documented companies are also the least active. US companies raise again at 49% against 26% elsewhere, partly because Crunchbase tracked them better.

Limitations

• Coverage. Crunchbase in 2013 was crowd-sourced and US-heavy (68% of rounds). Small and non-US rounds are under-reported, so “no next round” sometimes means “not recorded”.
• Missing signals. The model sees money and timing, not the team, product, traction or market, which is what investors actually judge. A ROC-AUC of 0.69 makes it a prioritisation aid, not a decision-maker.
• Age of the data. Funding behaviour in 2010–2011 is not the same as today’s. The method transfers; the learned weights don’t.

Next steps. Serve the saved pipeline behind an API, calibrate the probabilities, and add features for investor networks and sector momentum.

4.Conclusion and learnings

1. The label is a product decision. “Raises again eventually” and “raises again within 24 months” are different questions with different answers. Define the label as precisely as the business question.
2. Point-in-time or nothing. Every feature must be knowable on the prediction date. Snapshot totals are the most common way to leak the future. A quick test: add a suspect column back, and if the score leaps, it is leaking.
3. The split is part of the model. Split the way the model will be used: by time, and never with copies of the same entity on both sides.
4. Baselines first. A one-line rule scored 0.61. Without it, 0.69 has no meaning.
5. Tuning comes last. 18 settings moved the score by a thousandth. The data, the label and the split decide the result.
6. Serving is part of training. Save the whole pipeline, or the model can’t be used outside the notebook.

In traditional ML you own the model, so every one of these choices is visible in your own code. Generative models face the same problems under different names: benchmark contamination instead of leakage[2], eval sets instead of test splits. That is why evals are the hardest part of AI engineering, and why building one by hand is a good way to learn to write them.

References

1. Justinas Cirtautas. Startup Investments — Kaggle dataset of Crunchbase venture deals, organisations, people and exits. kaggle.com
2. Chunyuan Deng et al. Investigating Data Contamination in Modern Benchmarks for Large Language Models (2023). arxiv.org/abs/2311.09783