Notion Project Plan & Set Up Your Environment | Data Engineer Portfolio Project

A well-architected data warehouse requires a robust environment setup and disciplined project management. This module covers downloading source datasets (CRM and ERP), configuring essential developer tools (SQL Server Express, SSMS, Git, Draw.io), and building an Agile-style Notion project tracking system. By framing complex engineering tasks into structured Epics and Subtasks, you establish the foundation required to execute a high-stakes Data Engineering portfolio project successfully.

1. Project Files & Source Data Architecture

Every data warehousing project begins with raw data ingestion. For this portfolio build, the raw datasets originate from two distinct enterprise software operational systems: CRM (Customer Relationship Management) and ERP (Enterprise Resource Planning).

Source System Breakdown

• CRM System (Customer Relationship Management): Contains customer profiles, product catalog metadata, and direct sales transactions.
• ERP System (Enterprise Resource Planning): Captures enterprise-level operational metrics, including geographical location codes, historical customer mapping, and product category structures.
• File Distribution: Both systems provide three flat CSV files each, simulating a heterogeneous multi-source data ingestion pipeline into our analytical platform.

2. Essential Engineering Toolchain Setup

Building enterprise-grade data platforms requires specialized local database engines, management clients, diagramming interfaces, and planning boards.

3. The 50% Failure Rate: Why Project Planning Matters

Industry Reality & Mental Model: According to analytical industry reports (such as Gartner research), over 50% of Data Warehousing projects fail. The root cause is rarely syntax errors or infrastructure failures—it is a lack of clear architecture design, scope creep, and unstructured execution strategies.

The Naive vs. Structured Data Engineering Approach

Creating a project plan before writing code enforces clarity across the entire data lifecycle. Because full data architecture details evolve iteratively as data requirements unfold, engineers should start with a high-level, rough project roadmap, refining subtasks dynamically as structural understanding matures.

4. Building an Agile Project Plan in Notion

To manage complex data engineering builds, we adopt an Agile framework using Notion inline databases to structure execution into Epics and Tasks.

Concept Definitions

• Epic: A large, multi-stage project milestone representing a foundational phase (e.g., Requirements Analysis, Data Architecture Design).
• Task: A granular, actionable subtask belonging to an Epic (e.g., Choose Data Management Approach, Create Schema Scripts).

Step-by-Step Notion System Construction

1. Create the Master Epics Table:

   • Type /database inline on a blank Notion page.
   • Name the database Data Warehouse Epics.
   • Populate initial high-level phases:
     • Requirements Analysis
     • Design Data Architecture
     • Project Initialization

2. Create the Subtasks Table:

   • Create a second inline database named Data Warehouse Tasks.
   • Define columns for Task Name, Status Checkbox, and Epic Relation.

5. Linking Tables & Automated Progress Tracking

To visualize real-time project health, we establish a two-way relational link and apply dynamic rollup aggregations across database schemas.

Configuring the Two-Way Database Relation

1. Inside the Data Warehouse Tasks database, add a new property and select Relation.
2. Target the Data Warehouse Epics database and toggle Show on Data Warehouse Epics to enable two-way linking.
3. Assign every task to its parent Epic milestone.

Implementing the Dynamic Rollup Bar

1. In the Data Warehouse Epics database, create a new property and set its type to Rollup.
2. Select the existing relation: Data Warehouse Tasks.
3. Choose property target: Completed (Checkbox).
4. Set Calculate to Percent Checked.
5. Switch the display formatting from Number to Bar (or Ring).

Result: As individual checkboxes in the granular subtask board are completed, Notion automatically updates the parent Epic's percentage completion bar, providing visual confirmation of progress.

6. Notion UX Polish & Enterprise Equivalent Tools

To maximize scannability and modular management during complex technical builds, apply UX grouping directly within Notion:

1. Kanban/Grouped View: Open the options menu (...) on the Tasks table $\rightarrow$ Select Group $\rightarrow$ Group by Data Warehouse Epics.
2. Manual Sequence Sorting: Reorder parent epics sequentially to mirror your logical engineering workflow.
3. Visual Cues: Assign icon indicators to epics and tasks to denote current statuses.

Enterprise Tool Alignment

While enterprises typically use Jira, Azure DevOps, or Trello for team-based sprint planning, building a personal Notion planning framework demonstrates product ownership, architectural foresight, and project management maturity to hiring teams.

7. Initial Data Engineering Project Roadmap

With our planning environment established, the initial project lifecycle proceeds across three structured phases:

Initial Phase Breakdown

-- Phase 3 Target Output: Database Initialization SQL Script Preview
USE master;
GO

IF NOT EXISTS (SELECT * FROM sys.databases WHERE name = 'DataWarehouse')
BEGIN
    CREATE DATABASE DataWarehouse;
END;
GO

USE DataWarehouse;
GO

-- Create Schema Layers
CREATE SCHEMA bronze; -- Raw Data Ingestion Layer
GO
CREATE SCHEMA silver; -- Cleansed & Transformed Layer
GO
CREATE SCHEMA gold;   -- Business Analytical Layer
GO

Summary and Key Takeaways

• Multi-Source Data Architecture: The project relies on two operational system extractions—CRM and ERP—comprising 6 raw flat CSV files that will feed into the analytical platform.
• Complete Developer Toolchain: Successful deployment requires local server infrastructure (SQL Server Express), client IDE (SSMS), version control (Git/GitHub), visual diagramming software (Draw.io), and tracking platforms (Notion).
• The 50% Rule: Over half of enterprise data warehouse projects fail due to poor architectural planning and unclear execution boundaries. Pre-planning is an essential prerequisite for success.
• Agile Management Setup: Organizing engineering builds into Epics (major phases) and Tasks (actionable subtasks) ensures systematic progress.
• Relational Project Tracking: Linking tables with Two-Way Relations and Rollup Progress Bars in Notion gives real-time visual feedback on project progress.
• Structured Execution Strategy: Start with raw requirements, design architectural layers in Draw.io, set up Git repository standards, and execute DDL scripts to create SQL schemas in a controlled sequence.