Information Systems Integration course · Oct 2024
FC Porto squad data integration
A KNIME workflow that takes a Transfermarkt export of 32,405 players, isolates the FC Porto squad, cleans and joins player attributes with market valuations, and writes the result to CSV, JSON, XML and Excel, plus two charts.
Problem
Raw player exports are large and messy: a single CSV covers tens of thousands of players across every club, with gaps and fields that need joining to valuation history before they can answer anything about one squad.
Context and motivation
The course assignment was to apply data extraction, cleaning, transformation and loading with a visual workflow tool. I chose a football dataset and focused it on one squad.
What I built
Individual assignment: the KNIME workflow, its metanodes, the output files and the written report.
Technical decisions
2A visual, re-runnable pipeline
The whole process — filters, joiners, group-by, rankers and format writers — is a KNIME workflow, so it can be inspected node by node and re-executed rather than living in one-off scripts.
Clean once, export many times
The same cleaned table feeds every writer, so the CSV, JSON, XML and Excel outputs always contain the same data.
What it does
- Input: players.csv and player_valuations.csv
- Cleaning: missing-value filtering, row filtering and column ordering to isolate the FC Porto squad
- Transformation: normalisation, grouping and ranking, and a join between player attributes and valuations
- Output: the cleaned dataset as CSV, JSON, XML and XLSX
- Two charts: squad height distribution and squad market-value distribution
Evidence
what was checked, and how| What | Result | Source | Checked |
|---|---|---|---|
| Source data | CSV of 32,405 players from the Transfermarkt database | Project report (Information Systems Integration, IPCA 2024/25) | |
| Workflow | Committed and re-runnable in KNIME Analytics Platform | ProjetoISI/dataint in the repository |
Limitations and next steps
Known limitations
- A data-integration exercise: the charts are descriptive and the project does not draw scouting or performance conclusions.
- Market values and attributes are a snapshot from the source export and are not updated.