Bookmarks (#10)

* feat: add bookmarks table with CRUD operations and associated SQL queries

* ✨ feat: add bookmark management functionality (main)

- Update the bookmarks table to use UUID for user_id.
- Implement DAO and DTO for bookmark management.
- Introduce service layer handling CRUD operations for bookmarks.
- Add HTTP handlers for bookmarks feature in the server.
- Adjust SQL statements for using UUIDs in database operations.
- Modify similarity search threshold in assistant messages.

* ✨ feat: Update bookmark data structure and enhance fetching capabilities (main)

- Changed `summary_embeddings` and `content_embeddings` to optional fields in the SQL migration.
- Adjusted `BookmarkDTO` to use slice for embeddings instead of pgvector.
- Introduced content fetching on bookmark creation if content is not provided.
- Implemented the method to save bookmarks in the web summary handler.
- Updated HTTP handler to align with changes in create bookmark request structure.

* ✨ feat: Make embeddings column nullable for assistant embeddings (main)

- Add migrations to allow embeddings column to be nullable and update corresponding SQL files.
- Use pointer for Embeddings in DTO and DAO layers to handle potential nil values.
- Refactor database package to improve error handling and pointer usage for vector types.
- Update `sqlc.yaml` file to map nullable `vector` types to pointer in Go.

* 🌟 feat: Add bookmarks page with CRUD functionality

- Introduced a new `Bookmarks` page component allowing users to create, read, update, and delete bookmarks.
- Integrated with a protected route under the `/bookmarks` path.
- Utilized Mantine UI components for layout and interactions.
- Implemented data fetching and mutations using React Query.
- Enhanced the user interface with tooltips, icons, and modals for better UX.

* 🛠️ feat: add Browserless service support and enhance content processing (main)

- Introduced a new service for Browserless using Alpine Chrome.
- Enhanced the web content reading and processing capabilities by integrating markdown and summary processors.
- Updated dependencies in go.mod for improved functionality.
- Refined error handling for user authentication service.
- Added a new session management package for HTTP requests.

* 🎨 style: update bookmark summary to use MarkdownRenderer (main)

- Replace plain text rendering of bookmark summary with `MarkdownRenderer`.
- Add bookmark title display with `Title` component above content.

* ✨ feat: Update summary prompt and add language parameter support (main)

- Revise the summary prompt to align with the style of "The Wall Street Journal."
- Introduce a new parameter for language configuration in the summary processor.
- Implement templating with 'text/template' for dynamic prompt generation.
- Change the default language for summaries to Chinese.
- Update the model to 'gpt-4o-mini' as the default model.

* 🔧 refactor: Convert duration logging to milliseconds in time measurement (main)

- Updated duration logging to display time in milliseconds for consistency.
- Removed unnecessary import of `log/slog` from `middleware.go`.

* 🔨 refactor: update fetcher creation to use config options pattern (main)

- Replaced `NewBrowserFetcher` creation method to use options pattern.
- Replaced `NewHTTPFetcher` creation method with options pattern, adding options for timeout, max body size, redirects, retry count, and additional headers.
- Introduced `DefaultHTTPConfig`, `DefaultJinaConfig` for setting default configurations.
- Updated function names for consistency and clarity across the `WebReader` package.
- This facilitates easier configuration management and more flexible fetcher creation.

* ✨ feat: add bookmark refresh functionality (main)

- Introduced a new method `Refresh` in the `Service` for refreshing bookmarks.
  - This method allows fetching content from URLs using different fetchers and regenerating AI summaries.
  - Added a new interface `Summarier` and implemented it for processing summaries.
- Updated HTTP server handler to support bookmark refresh endpoint `POST /bookmarks/:bookmark-id/refresh`.
- Implemented front-end components and features for the refresh functionality.
  - Added `RefreshMenu` component and integrated it with the `BookmarkCard` and `BookmarkDetailModal`.
  - Created `refreshMutation` in React to handle refresh requests and manage UI updates.
- Made adjustments in `NewJinaFetcher` for error handling.

* ✨ feat: Add bookmarks and attachments endpoints and update docs (main)

- Introduced new endpoints for bookmarks including creating, updating, deleting, and refreshing bookmarks in the API documentation.
- Added new endpoints for listing attachments for assistants and threads.
- Updated data models to support the new API features, including `BookmarkDTO`, `RagSettings`, and related request schemes.
- Refactored UI components in `bookmarks.jsx` to align with the updated endpoints and functionalities.

database/migrations/000005_create_new_bookmarks_tables.down.sql

@@ -0,0 +1,5 @@
+DROP INDEX IF EXISTS idx_bookmarks_user_id;
+DROP INDEX IF EXISTS idx_bookmarks_url;
+DROP INDEX IF EXISTS idx_bookmarks_created_at;
+
+DROP TABLE IF EXISTS bookmarks;

database/migrations/000005_create_new_bookmarks_tables.up.sql

@@ -0,0 +1,20 @@
+CREATE TABLE IF NOT EXISTS bookmarks (
+    id SERIAL PRIMARY KEY,
+    uuid UUID UNIQUE NOT NULL DEFAULT gen_random_uuid(),
+    user_id UUID REFERENCES users(uuid),
+    url TEXT NOT NULL,
+    title VARCHAR(255),
+    summary TEXT,
+    summary_embeddings vector(1536),
+    content TEXT,
+    content_embeddings vector(1536),
+    html TEXT,
+    metadata JSONB,
+    screenshot TEXT,
+    created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
+);
+
+CREATE INDEX IF NOT EXISTS idx_user_created_at ON bookmarks (user_id, updated_at);
+CREATE INDEX IF NOT EXISTS idx_uuid ON bookmarks (uuid);
+CREATE INDEX IF NOT EXISTS idx_url ON bookmarks (url);

database/migrations/000006_set_embeddings_to_nullable.down.sql

@@ -0,0 +1 @@
+ALTER TABLE assistant_embedddings ALTER COLUMN embeddings TYPE vector(1536), ALTER COLUMN embeddings SET NOT NULL;

database/migrations/000006_set_embeddings_to_nullable.up.sql

@@ -0,0 +1 @@
+ALTER TABLE assistant_embedddings ALTER COLUMN embeddings TYPE vector(1536), ALTER COLUMN embeddings DROP NOT NULL;

database/queries/bookmarks.sql

@@ -0,0 +1,49 @@
+-- name: CreateBookmark :one
+INSERT INTO bookmarks (
+    uuid,
+    user_id,
+    url,
+    title,
+    summary,
+    summary_embeddings,
+    content,
+    content_embeddings,
+    html,
+    metadata,
+    screenshot
+) VALUES (
+    $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11
+) RETURNING *;
+
+-- name: GetBookmarkByUUID :one
+SELECT * FROM bookmarks WHERE uuid = $1;
+
+-- name: GetBookmarkByURL :one
+SELECT * FROM bookmarks WHERE url = $1 AND user_id = $2;
+
+-- name: ListBookmarks :many
+SELECT * FROM bookmarks 
+WHERE user_id = $1 
+ORDER BY updated_at DESC 
+LIMIT $2 OFFSET $3;
+
+-- name: UpdateBookmark :one
+UPDATE bookmarks 
+SET 
+    title = COALESCE($3, title),
+    summary = COALESCE($4, summary),
+    summary_embeddings = COALESCE($5, summary_embeddings),
+    content = COALESCE($6, content),
+    content_embeddings = COALESCE($7, content_embeddings),
+    html = COALESCE($8, html),
+    metadata = COALESCE($9, metadata),
+    screenshot = COALESCE($10, screenshot),
+    updated_at = CURRENT_TIMESTAMP
+WHERE uuid = $1 AND user_id = $2
+RETURNING *;
+
+-- name: DeleteBookmark :exec
+DELETE FROM bookmarks WHERE uuid = $1 AND user_id = $2;
+
+-- name: DeleteBookmarksByUser :exec
+DELETE FROM bookmarks WHERE user_id = $1;

docker-compose.yml

@@ -18,6 +18,18 @@ services:
       - .env:/app/.env
     depends_on:
       - postgres
+  browserless:
+    image:  gcr.docker.vaayne.com/zenika-hub/alpine-chrome:123
+    restart: unless-stopped
+    ports:
+      - "9222:9222"
+    command:
+      - --no-sandbox
+      - --disable-gpu
+      - --disable-dev-shm-usage
+      - --remote-debugging-address=0.0.0.0
+      - --remote-debugging-port=9222
+      # - --hide-scrollbars
   postgres:
     image: pgvector/pgvector:pg16
     environment: