StackOverflow Java Stats

A comprehensive full-stack data visualization dashboard designed to analyze and display insights from over 7,500 StackOverflow threads tagged with Java. This project explores topic trends, tag co-occurrences, common multithreading pitfalls, and factors affecting question solvability.

📺 Preview

so-java-stats.mp4

🛠 Tech Stack

Frontend

Backend

📂 Project Structure

.
├── backend/so-java-stats
│   ├── Dockerfile
│   ├── pom.xml
│   ├── src/main
│   │   ├── java/com/sustech/so_java_stats
│   │   │   ├── SoJavaStatsApplication.java
│   │   │   ├── config/                 # CORS and OpenAPI/Swagger config
│   │   │   ├── controller/             # REST API Endpoints
│   │   │   ├── dto/                    # Data Transfer Objects for API responses
│   │   │   ├── init/                   # Database Initializer (JSON to DB logic)
│   │   │   ├── model/                  # JPA Entities (Question, Answer, Tag, etc.)
│   │   │   ├── repository/             # Spring Data JPA Repositories
│   │   │   └── service/                # Business Logic interfaces and implementations
│   │   └── resources
│   │       ├── application.yaml        # App configuration
│   │       └── data/                   # raw data source [7500 thread_X.json files]
│   └── mvnw
├── frontend
│   ├── Dockerfile
│   ├── index.html
│   ├── nginx.conf                      # Production server config
│   ├── package.json
│   ├── postcss.config.js
│   ├── tailwind.config.js              # Styling configuration
│   ├── vite.config.js                  # Build tool configuration
│   └── src
│       ├── App.jsx                     # Main React component
│       ├── main.jsx                    # Entry point
│       ├── components
│       │   ├── QuestionModal.jsx
│       │   └── charts/                 # Dashboard Visualization components
│       │       ├── MultithreadingPitfalls.jsx
│       │       ├── QuestionSolvability.jsx
│       │       ├── TopicCooccurrences.jsx
│       │       └── TopicTrends.jsx
│       ├── utils/                      # Export/Download utilities
│       index.css
├── data-scraper
│   ├── pom.xml
│   └── src/main/java/DataScraper.java  # Scraper entry point
├── docker-compose.yaml
└── README.md

📊 Analytics Features

• Topic Trends: A time-series analysis showing the volume of questions for specific Java sub-topics (e.g., Spring, JVM, Multithreading) over time.

• Topic Co-occurrences: A relationship graph/chart identifying which technologies are most frequently paired with Java.

• Concurrency Analysis: A regex-based deep dive into question bodies and answers to identify the most common multithreading "pitfalls" (Deadlocks, Race Conditions, etc.).

• Solvability Analysis: Comparison of metrics between "Solvable" (accepted answer) and "Hard-to-solve" questions, looking at user reputation and body length.

🕷 Data Scraper

The data-scraper module is a standalone Java tool that interfaces with the StackExchange API. It uses a custom filter to fetch rich thread data (including owners, comments, and full answer bodies) and saves them as serialized JSON files for the backend to ingest.

🚀 Getting Started

Prerequisites

• Docker and Docker Compose (Recommended)

OR Local Environment:

• Java 17+
• Node.js 18+
• PostgreSQL 15+

Option 1: Using Docker (Fastest)

1. Clone the repository.

2. Create a .env file in the root directory with the following variables:

DB_USERNAME=your_username
DB_PASSWORD=your_password

3. Run the following command:

docker-compose up -d --build

4. Access the dashboard at http://localhost:3000.

Option 2: Local Development Setup

1. Database

   Create a PostgreSQL database named stackoverflow.

   Ensure your local credentials match your .env or application.yaml.

2. Backend

   Navigate to backend/so-java-stats.

   Create a .env file there with your DB_USERNAME and DB_PASSWORD.

   Run the application:

   ./mvnw spring-boot:run
   

   Note: On first run, the DatabaseInitializer will automatically parse the 7,500 JSON files in resources/data and populate your PostgreSQL database.

3. Frontend

   Navigate to frontend.

   Install dependencies and start the dev server:

   npm install
   npm run dev
   

   Access the app at http://localhost:5173.