Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/build model graph #989

Draft
wants to merge 48 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
5a6906e
commit
yashgoyal0110 Mar 1, 2025
6d6feb5
commit
yashgoyal0110 Mar 2, 2025
27b5b06
commit
yashgoyal0110 Mar 2, 2025
3e703fb
commit
yashgoyal0110 Mar 2, 2025
266db96
commit
yashgoyal0110 Mar 2, 2025
85c49b9
commit
yashgoyal0110 Mar 2, 2025
557d498
commit
yashgoyal0110 Mar 2, 2025
70e104f
commit
yashgoyal0110 Mar 2, 2025
2a6580d
commit
yashgoyal0110 Mar 2, 2025
ab139a0
commit
yashgoyal0110 Mar 2, 2025
84a404a
commit
yashgoyal0110 Mar 2, 2025
d262005
commit
yashgoyal0110 Mar 2, 2025
b6244d7
commit
yashgoyal0110 Mar 2, 2025
0a5140d
commit
yashgoyal0110 Mar 2, 2025
11cba6e
commit
yashgoyal0110 Mar 2, 2025
13bc93c
Commit
yashgoyal0110 Mar 2, 2025
d49643b
Commit
yashgoyal0110 Mar 2, 2025
df31e8d
Commit
yashgoyal0110 Mar 2, 2025
6c52f16
Commit
yashgoyal0110 Mar 2, 2025
5908048
Commit
yashgoyal0110 Mar 2, 2025
77d1025
Commit
yashgoyal0110 Mar 2, 2025
784b8ed
Commit
yashgoyal0110 Mar 2, 2025
15ba619
Commit
yashgoyal0110 Mar 2, 2025
664cc77
Commit
yashgoyal0110 Mar 2, 2025
801d5e8
Commit
yashgoyal0110 Mar 2, 2025
19d1f78
Commit
yashgoyal0110 Mar 2, 2025
4ac3dba
Commit
yashgoyal0110 Mar 2, 2025
700bf15
Commit
yashgoyal0110 Mar 2, 2025
8f60cbb
Commit
yashgoyal0110 Mar 2, 2025
f5d83ec
Commit
yashgoyal0110 Mar 2, 2025
240068a
Commit
yashgoyal0110 Mar 2, 2025
a2dc821
Commit
yashgoyal0110 Mar 2, 2025
4280f2f
Commit
yashgoyal0110 Mar 2, 2025
448eae9
Commit
yashgoyal0110 Mar 2, 2025
023a2c2
Commit
yashgoyal0110 Mar 2, 2025
8a2b81f
inventory
yashgoyal0110 Mar 3, 2025
7ad7315
commit
yashgoyal0110 Mar 3, 2025
cf24c10
commit
yashgoyal0110 Mar 3, 2025
df949e9
commit
yashgoyal0110 Mar 3, 2025
f15b224
commit
yashgoyal0110 Mar 3, 2025
ed6c4d1
Merge branch 'main' into feat/build-model-graph
yashgoyal0110 Mar 3, 2025
1398fa3
commit
yashgoyal0110 Mar 3, 2025
74b3388
commit
yashgoyal0110 Mar 3, 2025
2258061
Merge branch 'OWASP:main' into feat/build-model-graph
yashgoyal0110 Mar 4, 2025
be78e4a
Merge branch 'main' into feat/build-model-graph
yashgoyal0110 Mar 7, 2025
58efa99
Merge branch 'main' into feat/build-model-graph
yashgoyal0110 Mar 7, 2025
cf16d6b
comit
yashgoyal0110 Mar 8, 2025
a69e7ce
Merge branch 'main' into feat/build-model-graph
yashgoyal0110 Mar 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions .github/workflows/generate_erd.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: Generate Models Graph

on:
push:
paths:
- "backend/apps/**/models/**"
- "pyproject.toml"
- "README.md"
workflow_dispatch:

jobs:
generate-model-graph:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.13"

- name: Install Poetry
run: |
python -m pip install --upgrade pip
python -m pip install poetry

- name: Install system dependencies
run: |
sudo apt update
sudo apt install -y graphviz

- name: Create and Populate .env File
run: |
touch backend/.env # Ensure the file exists
echo "DJANGO_SENTRY_DSN=${{ secrets.DJANGO_SENTRY_DSN }}" >> backend/.env
echo "DJANGO_SECRET_KEY=${{ secrets.DJANGO_SECRET_KEY }}" >> backend/.env
echo "SECRET_KEY=${{ secrets.SECRET_KEY }}" >> backend/.env
echo "DJANGO_CONFIGURATION=Test" >> backend/.env
echo "GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}" >> backend/.env
echo "DJANGO_DB_PASSWORD=${{ secrets.DJANGO_DB_PASSWORD }}" >> backend/.env
echo "DJANGO_ALLOWED_HOSTS=*" >> backend/.env # Example of a non-secret variable
Comment on lines +36 to +43
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look right. You shouldn't need the real secrets for building a model relations graph.

Comment on lines +34 to +43
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consolidate Environment Variable Configuration
Two different environment files are being created in subsequent steps. Consolidating all environment variables into a single file (for example, using only backend/.env) would improve clarity and reduce risks of misconfiguration or accidental exposure.


- name: Debug .env file
run: |
ls -la backend # Check if the .env file exists
cat backend/.env | grep -v DJANGO_DB_PASSWORD | grep -v DJANGO_SECRET_KEY

Comment on lines +34 to +49
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Secure the environment variable handling

There are several concerns with the current environment variable handling:

  1. Debugging environment variables can potentially expose sensitive information
  2. The grep filter might not catch all sensitive data
  3. Multiple .env files are being created, which can lead to confusion

Remove the debug step completely and consolidate environment variable handling:

- - name: Create and Populate .env File
-   run: |
-     touch backend/.env  # Ensure the file exists
-     echo "DJANGO_SENTRY_DSN=${{ secrets.DJANGO_SENTRY_DSN }}" >> backend/.env
-     echo "DJANGO_SECRET_KEY=${{ secrets.DJANGO_SECRET_KEY }}" >> backend/.env
-     echo "SECRET_KEY=${{ secrets.SECRET_KEY }}" >> backend/.env
-     echo "DJANGO_CONFIGURATION=Test" >> backend/.env
-     echo "GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}" >> backend/.env
-     echo "DJANGO_DB_PASSWORD=${{ secrets.DJANGO_DB_PASSWORD }}" >> backend/.env
-     echo "DJANGO_ALLOWED_HOSTS=*" >> backend/.env  # Example of a non-secret variable

- - name: Debug .env file
-   run: |
-     ls -la backend  # Check if the .env file exists
-     cat backend/.env | grep -v DJANGO_DB_PASSWORD | grep -v DJANGO_SECRET_KEY
+ - name: Create environment configuration
+   run: |
+     # Create a single .env file for the workflow
+     touch backend/.env
+     echo "DJANGO_SENTRY_DSN=${{ secrets.DJANGO_SENTRY_DSN }}" >> backend/.env
+     echo "DJANGO_SECRET_KEY=${{ secrets.DJANGO_SECRET_KEY }}" >> backend/.env
+     echo "SECRET_KEY=${{ secrets.SECRET_KEY }}" >> backend/.env
+     echo "DJANGO_CONFIGURATION=Test" >> backend/.env
+     echo "DJANGO_SETTINGS_MODULE=settings.test" >> backend/.env
+     echo "GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}" >> backend/.env
+     echo "DJANGO_DB_PASSWORD=${{ secrets.DJANGO_DB_PASSWORD }}" >> backend/.env
+     echo "DJANGO_ALLOWED_HOSTS=*" >> backend/.env
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Create and Populate .env File
run: |
touch backend/.env # Ensure the file exists
echo "DJANGO_SENTRY_DSN=${{ secrets.DJANGO_SENTRY_DSN }}" >> backend/.env
echo "DJANGO_SECRET_KEY=${{ secrets.DJANGO_SECRET_KEY }}" >> backend/.env
echo "SECRET_KEY=${{ secrets.SECRET_KEY }}" >> backend/.env
echo "DJANGO_CONFIGURATION=Test" >> backend/.env
echo "GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}" >> backend/.env
echo "DJANGO_DB_PASSWORD=${{ secrets.DJANGO_DB_PASSWORD }}" >> backend/.env
echo "DJANGO_ALLOWED_HOSTS=*" >> backend/.env # Example of a non-secret variable
- name: Debug .env file
run: |
ls -la backend # Check if the .env file exists
cat backend/.env | grep -v DJANGO_DB_PASSWORD | grep -v DJANGO_SECRET_KEY
- name: Create environment configuration
run: |
# Create a single .env file for the workflow
touch backend/.env
echo "DJANGO_SENTRY_DSN=${{ secrets.DJANGO_SENTRY_DSN }}" >> backend/.env
echo "DJANGO_SECRET_KEY=${{ secrets.DJANGO_SECRET_KEY }}" >> backend/.env
echo "SECRET_KEY=${{ secrets.SECRET_KEY }}" >> backend/.env
echo "DJANGO_CONFIGURATION=Test" >> backend/.env
echo "DJANGO_SETTINGS_MODULE=settings.test" >> backend/.env
echo "GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}" >> backend/.env
echo "DJANGO_DB_PASSWORD=${{ secrets.DJANGO_DB_PASSWORD }}" >> backend/.env
echo "DJANGO_ALLOWED_HOSTS=*" >> backend/.env

Comment on lines +45 to +49
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Remove Debug Step to Protect Sensitive Information
The "Debug .env file" step displays the contents of the .env file (even with filters), which might still expose sensitive data such as SECRET_KEY and GITHUB_TOKEN. It is recommended to remove this step (or restrict its execution strictly to development) to prevent potential leakage.

- name: Prepare secrets
run: |
echo "DJANGO_SENTRY_DSN=${{ secrets.DJANGO_SENTRY_DSN }}" >> .env.backend
echo "DJANGO_SECRET_KEY=${{ secrets.DJANGO_SECRET_KEY }}" >> backend/.env.backend
echo "DJANGO_SETTINGS_MODULE=settings.test" >> backend/.env.backend
echo "DJANGO_CONFIGURATION=Test"
echo "SECRET_KEY=${{ secrets.SECRET_KEY }}" >> backend/.env
echo "GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}" >> backend/.env.backend

Comment on lines +50 to +58
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Remove Redundant “Prepare Secrets” Step
This step creates an additional .env.backend file while you already have a .env file populated earlier. Consolidating environment variable handling into a single file not only minimizes redundancy but also reduces the risk of configuration mismatches or leaks.

- name: Install dependencies
run: |
cd backend
export $(grep -v '^#' .env.backend | xargs)
poetry install --no-interaction --no-root --with dev
poetry add pydotplus # Ensure pydotplus is installed

- name: Generate Models Graph
run: |
cd backend
set -a
source .env.backend
set +a
poetry run python manage.py graph_models -a -g -o backend_models.png
Comment on lines +68 to +72
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Update environment variable sourcing for clarity

Since we're consolidating to a single .env file in the previous recommendations, the environment variable sourcing needs to be updated.

- set -a
- source .env.backend
- set +a
+ # Load environment variables from consolidated .env file
+ set -a
+ source .env
+ set +a
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
cd backend
set -a
source .env.backend
set +a
poetry run python manage.py graph_models -a -g -o backend_models.png
cd backend
# Load environment variables from consolidated .env file
set -a
source .env
set +a
poetry run python manage.py graph_models -a -g -o backend_models.png


Comment on lines +66 to +73
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Use a Consolidated Environment File for Graph Generation
This step sources .env.backend whereas the earlier step creates and populates backend/.env for environment variables. Using multiple files can lead to configuration mismatches. Align these steps by using a single, consolidated environment file.

- name: Upload Artifact
uses: actions/upload-artifact@v4
with:
name: backend-models-graph
path: backend/backend_models.png
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,14 @@ OWASP Nest thrives on community contributions. Whether you are a developer, desi
- Issue Reporting: Identify and report bugs or suggest improvements.
- Engagement: Share feedback, participate in discussions, or promote the project in your network.

## Django Models Graph

The following is an auto-generated visualization of our Django models:

![Django Models Graph](backend/backend_models.png)



To get started, visit the [OWASP Nest Repository](https://github.com/OWASP/Nest), explore the [Contributing Guidelines](https://github.com/OWASP/Nest/blob/main/CONTRIBUTING.md), and [Code of Conduct](https://github.com/OWASP/Nest/blob/main/CODE_OF_CONDUCT.md).

## About
Expand Down
Binary file added backend/backend_models.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 4 additions & 2 deletions backend/manage.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@
import sys

if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings.local")
os.environ.setdefault("DJANGO_CONFIGURATION", "Local")
os.environ.setdefault(
"DJANGO_SETTINGS_MODULE", os.getenv("DJANGO_SETTINGS_MODULE", "settings.test")
)
os.environ.setdefault("DJANGO_CONFIGURATION", os.getenv("DJANGO_CONFIGURATION", "Test"))

from configurations.management import execute_from_command_line

Expand Down
54 changes: 49 additions & 5 deletions backend/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions backend/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,13 @@ slack-bolt = "^1.22.0"


[tool.poetry.group.dev.dependencies]
django-extensions = "^3.2.3"
pydotplus = "^2.0.2"
djlint = "^1.36.4"
pre-commit = "^4.1.0"
ruff = "^0.9.5"


[tool.poetry.group.test.dependencies]
pytest = "^8.3.4"
pytest-cov = "^6.0"
Expand Down
2 changes: 1 addition & 1 deletion backend/settings/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ class Base(Configuration):
DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = values.SecretValue()
SECRET_KEY = values.Value()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Revert SECRET_KEY to SecretValue for security reasons

Changing SECRET_KEY from values.SecretValue() to values.Value() reduces security by potentially exposing the secret key in debug outputs, logs, and error messages. SecretValue is specifically designed to handle sensitive information like secret keys.

-    SECRET_KEY = values.Value()
+    SECRET_KEY = values.SecretValue()

This maintains the appropriate security handling for this sensitive configuration value.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
SECRET_KEY = values.Value()
SECRET_KEY = values.SecretValue()


# https://docs.djangoproject.com/en/5.1/ref/settings/#data-upload-max-number-fields
DATA_UPLOAD_MAX_NUMBER_FIELDS = 5000
Expand Down
2 changes: 2 additions & 0 deletions backend/settings/local.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,5 @@ class Local(Base):
LOGGING = {}
SLACK_COMMANDS_ENABLED = True
SLACK_EVENTS_ENABLED = True

INSTALLED_APPS = ("django_extensions",)
3 changes: 3 additions & 0 deletions backend/settings/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@ class Test(Base):
"""Test configuration."""

DEBUG = False


INSTALLED_APPS = ("django_extensions",)
Loading