feat: display local pdf files

[thucpn]

Summary by CodeRabbit

• New Features

  • Introduced an API endpoint to retrieve file data based on a provided path.
  • Added PdfDialog component to display PDF content within a drawer.
  • Enhanced ChatSources component to aggregate and display nodes by URL or file path.

• Enhancements

  • Updated ChatSources rendering logic to differentiate between PDFs and other sources.

• Dependencies

  • Added vaul version 0.9.1 and @llamaindex/pdf-viewer version 1.1.1 to dependencies.

• Backend

  • Enabled serving of static files from the /data endpoint in FastAPI.

[changeset-bot]

🦋 Changeset detected

Latest commit: d54fe0a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
create-llama	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[coderabbitai]

Walkthrough

This update introduces a new API endpoint for fetching file data, enhances the chat component to handle and display PDF files, and includes support for serving static files in a FastAPI application. Additionally, new dependencies have been added to facilitate these features.

Changes

File(s)	Change Summary
templates/types/streaming/nextjs/app/api/data/[path]/route.ts	Introduces an API endpoint for retrieving file data from the ./data folder based on the provided path slug.
templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx	Enhances the ChatSources component to handle and display PDF files, includes new imports, and updates the rendering logic.
templates/types/streaming/nextjs/app/components/ui/chat/widgets/PdfDialog.tsx	Introduces the PdfDialog component for displaying PDF content within a drawer.
templates/types/streaming/nextjs/app/components/ui/lib/url.ts	Adds a function getStaticFileDataUrl to generate a URL for file data based on the input filename.
templates/types/streaming/nextjs/package.json	Adds new dependencies vaul and @llamaindex/pdf-viewer.
templates/types/streaming/fastapi/main.py	Adds serving of static files from the /data endpoint using app.mount.

In bytes and bits, we weave our tale,
With endpoints new, our code sets sail.
PDFs now within our reach,
In drawers snug, they do beseech.
FastAPI serves with grace and flair,
Data flows through springtime air.
🐇✨

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

Actionable comments posted: 1

Out of diff range and nitpick comments (2)

templates/types/streaming/express/index.ts (1)

34-34: Consider documenting the contents of the /data directory.

It's a good practice to document what types of files are expected to be in the /data directory to prevent accidental exposure of sensitive files.

templates/types/streaming/fastapi/main.py (1)

41-41: Consider documenting the contents of the /data directory.

As with any public file serving, documenting the expected contents of the /data directory can help prevent the accidental exposure of sensitive files.

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 78776ac and 45d66af.

Files selected for processing (6)

• templates/types/streaming/express/index.ts (1 hunks)
• templates/types/streaming/fastapi/main.py (3 hunks)
• templates/types/streaming/nextjs/app/api/data/route.ts (1 hunks)
• templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx (4 hunks)
• templates/types/streaming/nextjs/app/components/ui/drawer.tsx (1 hunks)
• templates/types/streaming/nextjs/package.json (1 hunks)

Additional Context Used

Ruff (10)

templates/types/streaming/fastapi/main.py (10)

5-5: Module level import not at top of file

---

6-6: Module level import not at top of file

---

7-7: Module level import not at top of file

---

8-8: Module level import not at top of file

---

9-9: Module level import not at top of file

---

10-10: Module level import not at top of file

---

11-11: Module level import not at top of file

---

12-12: Module level import not at top of file

---

13-13: Module level import not at top of file

---

14-14: Module level import not at top of file

Additional comments not posted (3)

templates/types/streaming/nextjs/package.json (1)

36-36: Addition of PDF viewer library aligns with new features.

The inclusion of @llamaindex/pdf-viewer is appropriate for the new PDF handling capabilities introduced in the project.

templates/types/streaming/nextjs/app/components/ui/drawer.tsx (1)

1-118: Drawer components are well-implemented and structured.

The use of vaul library components, along with React.forwardRef and proper use of displayName, enhances the maintainability and debuggability of the UI components.

templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx (1)

Line range hint 1-153: Verify the consistent and secure use of environment variables.

Ensure that environment variables are used consistently and securely throughout the project.

Verification successful

---

The verification of the usage of environment variables across the project has been successful. The results show that environment variables are used consistently and securely throughout various components and settings files. The usage patterns include checks for the existence of variables, default values, and secure handling of potentially undefined variables.

Here are some key points observed from the output:

• Environment variables are used in both frontend (nextjs) and backend (express) components.
• Variables are accessed securely with fallbacks or checks (e.g., process.env.MODEL ?? "gpt-3.5-turbo").
• Critical configurations like API endpoints and database connections are controlled via environment variables, which is a best practice for maintaining security and flexibility.

Given these observations, the use of environment variables in the project aligns with common practices for secure and flexible configuration management.

Scripts Executed

The following scripts were executed for the analysis:

---

Script:

#!/bin/bash
# Description: Verify the usage of environment variables in the Next.js project.

# Test: Search for environment variable usage. Expect: Consistent and secure usage.
rg --type typescript $'process.env'

Length of output: 69

---

---

Script:

#!/bin/bash
# Description: Verify the usage of environment variables in the Next.js project, correcting the file type issue.

# Define 'typescript' as a type including .ts and .tsx files
rg --type-add 'typescript:*.ts' --type-add 'typescript:*.tsx' --type typescript 'process.env'

Length of output: 8482

[coderabbitai]

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 45d66af and 021bd3f.

Files selected for processing (1)

• .changeset/weak-bobcats-trade.md (1 hunks)

Files skipped from review due to trivial changes (1)

• .changeset/weak-bobcats-trade.md

[coderabbitai]

Actionable comments posted: 5

Out of diff range and nitpick comments (2)

templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx (2)

Line range hint 39-93: Consider memoizing the getNodeInfo function and moving the PdfDialog component to its own file.

To improve performance and maintainability, consider memoizing the getNodeInfo function and moving the PdfDialog component to its own file. This will make the codebase more modular and easier to navigate.

import { useMemo, useCallback } from "react";
import PdfDialog from "./widgets/PdfDialog";

const SCORE_THRESHOLD = 0.5;

function SourceNumberButton({ index }: { index: number }) {
  return (
    <div className="text-xs w-5 h-5 rounded-full bg-gray-100 mb-2 flex items-center justify-center hover:text-white hover:bg-primary hover:cursor-pointer">
      {index + 1}
    </div>
  );
}

enum NODE_TYPE {
  URL,
  LOCAL_FILE,
}

const getNodeInfo = useCallback((node: SourceNode): { path: string; type: NODE_TYPE } | undefined => {
  if (typeof node.metadata["URL"] === "string") {
    return {
      path: node.metadata["URL"],
      type: NODE_TYPE.URL,
    };
  }
  if (typeof node.metadata["file_path"] === "string") {
    return {
      path: node.metadata["file_path"],
      type: NODE_TYPE.LOCAL_FILE,
    };
  }
  return undefined;
}, []);

export function ChatSources({ data }: { data: SourceData }) {
  const sources = useMemo(() => {
    // aggregate nodes by url or file_path (get the highest one by score)
    const nodePaths = new Set<string>();
    const uniqueNodes: SourceNode[] = [];

    data.nodes
      .sort((a, b) => (b.score ?? 1) - (a.score ?? 1))
      .forEach((node) => {
        const nodeInfo = getNodeInfo(node);
        if (!nodeInfo) {
          uniqueNodes.push(node); // Always add nodes with unknown type
        } else if (!nodePaths.has(nodeInfo.path)) {
          uniqueNodes.push(node);
          nodePaths.add(nodeInfo.path);
        }
      });

    return uniqueNodes
      .filter((node) => Object.keys(node.metadata).length > 0)
      .filter((node) => (node.score ?? 1) > SCORE_THRESHOLD);
  }, [data.nodes, getNodeInfo]);

  if (sources.length === 0) return null;

  return (
    <div className="space-x-2 text-sm">
      <span className="font-semibold">Sources:</span>
      <div className="inline-flex gap-1 items-center">
        {sources.map((node: SourceNode, index: number) => {
          const nodeInfo = getNodeInfo(node);
          if (nodeInfo?.path.endsWith(".pdf")) {
            return (
              <PdfDialog
                key={node.id}
                documentId={node.id}
                filePath={nodeInfo.path}
                trigger={<SourceNumberButton index={index} />}
              />
            );
          }
          return (
            <div key={node.id}>
              <HoverCard>
                <HoverCardTrigger>
                  <SourceNumberButton index={index} />
                </HoverCardTrigger>
                <HoverCardContent className="w-[320px]">
                  <NodeInfo node={node} />
                </HoverCardContent>
              </HoverCard>
            </div>
          );
        })}
      </div>
    </div>
  );
}

function NodeInfo({ node }: { node: SourceNode }) {
  const { isCopied, copyToClipboard } = useCopyToClipboard({ timeout: 1000 });
  const nodeInfo = getNodeInfo(node);

  if (nodeInfo?.type === NODE_TYPE.URL) {
    // this is a node generated by the web loader, it contains an external URL
    // add a link to view this URL
    return (
      <a
        className="space-x-2 flex items-center my-2 hover:text-blue-900"
        href={nodeInfo.path}
        target="_blank"
      >
        <span>{nodeInfo.path}</span>
        <ArrowUpRightSquare className="w-4 h-4" />
      </a>
    );
  }

  if (nodeInfo?.type === NODE_TYPE.LOCAL_FILE) {
    // this is a node generated by the file loader, it contains file path
    // add a button to copy the path to the clipboard
    return (
      <div className="flex items-center justify-between gap-2">
        <span className="flex-1 break-all">{nodeInfo.path}</span>
        <Button
          onClick={() => copyToClipboard(nodeInfo.path)}
          size="icon"
          variant="ghost"
          className="h-12 w-12 shrink-0"
        >
          {isCopied ? (
            <Check className="h-4 w-4" />
          ) : (
            <Copy className="h-4 w-4" />
          )}
        </Button>
      </div>
    );
  }

  // node generated by unknown loader, implement renderer by analyzing logged out metadata
  console.log("Node metadata", node.metadata);
  return (
    <p>
      Sorry, unknown node type. Please add a new renderer in the NodeInfo
      component.
    </p>
  );
}

---

Line range hint 100-135: Consider adding type annotations and comments for the NodeInfo component.

The NodeInfo component is correctly implemented, but consider adding type annotations and comments to describe each part of the component for better clarity and maintainability.

function NodeInfo({ node }: { node: SourceNode }) {
  const { isCopied, copyToClipboard } = useCopyToClipboard({ timeout: 1000 });
  const nodeInfo = getNodeInfo(node);

  if (nodeInfo?.type === NODE_TYPE.URL) {
    // this is a node generated by the web loader, it contains an external URL
    // add a link to view this URL
    return (
      <a
        className="space-x-2 flex items-center my-2 hover:text-blue-900"
        href={nodeInfo.path}
        target="_blank"
      >
        <span>{nodeInfo.path}</span>
        <ArrowUpRightSquare className="w-4 h-4" />
      </a>
    );
  }

  if (nodeInfo?.type === NODE_TYPE.LOCAL_FILE) {
    // this is a node generated by the file loader, it contains file path
    // add a button to copy the path to the clipboard
    return (
      <div className="flex items-center justify-between gap-2">
        <span className="flex-1 break-all">{nodeInfo.path}</span>
        <Button
          onClick={() => copyToClipboard(nodeInfo.path)}
          size="icon"
          variant="ghost"
          className="h-12 w-12 shrink-0"
        >
          {isCopied ? (
            <Check className="h-4 w-4" />
          ) : (
            <Copy className="h-4 w-4" />
          )}
        </Button>
      </div>
    );
  }

  // node generated by unknown loader, implement renderer by analyzing logged out metadata
  console.log("Node metadata", node.metadata);
  return (
    <p>
      Sorry, unknown node type. Please add a new renderer in the NodeInfo
      component.
    </p>
  );
}

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 021bd3f and 56ed2f8.

Files selected for processing (3)

• .changeset/weak-bobcats-trade.md (1 hunks)
• templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx (2 hunks)
• templates/types/streaming/nextjs/app/components/ui/chat/widgets/PdfDialog.tsx (1 hunks)

Files skipped from review as they are similar to previous changes (1)

• .changeset/weak-bobcats-trade.md

[coderabbitai]

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 56ed2f8 and ec9a97f.

Files selected for processing (1)

• templates/types/streaming/nextjs/app/api/data/route.ts (1 hunks)

Files skipped from review as they are similar to previous changes (1)

• templates/types/streaming/nextjs/app/api/data/route.ts

[coderabbitai]

Actionable comments posted: 2

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between ec9a97f and 8a2585c.

Files selected for processing (4)

• templates/types/streaming/nextjs/app/api/data/[path]/route.ts (1 hunks)
• templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx (2 hunks)
• templates/types/streaming/nextjs/app/components/ui/chat/widgets/PdfDialog.tsx (1 hunks)
• templates/types/streaming/nextjs/next.config.mjs (1 hunks)

Files skipped from review as they are similar to previous changes (2)

• templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx
• templates/types/streaming/nextjs/app/components/ui/chat/widgets/PdfDialog.tsx

Additional comments not posted (1)

templates/types/streaming/nextjs/next.config.mjs (1)

8-16: Ensure the /api/data/:path* endpoint is correctly implemented.

The rewrites configuration looks correct. Please verify that the /api/data/:path* endpoint is correctly implemented and handles the requests as expected.

[coderabbitai]

Actionable comments posted: 5

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 8a2585c and df77dd5.

Files selected for processing (3)

• templates/types/streaming/express/index.ts (1 hunks)
• templates/types/streaming/fastapi/main.py (3 hunks)
• templates/types/streaming/nextjs/app/components/ui/lib/url.ts (1 hunks)

Files skipped from review as they are similar to previous changes (1)

• templates/types/streaming/express/index.ts

Additional Context Used

Ruff (10)

templates/types/streaming/fastapi/main.py (10)

5-5: Module level import not at top of file

---

6-6: Module level import not at top of file

---

7-7: Module level import not at top of file

---

8-8: Module level import not at top of file

---

9-9: Module level import not at top of file

---

10-10: Module level import not at top of file

---

11-11: Module level import not at top of file

---

12-12: Module level import not at top of file

---

13-13: Module level import not at top of file

---

14-14: Module level import not at top of file

[coderabbitai]

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between df77dd5 and b4b049c.

Files selected for processing (4)

• templates/types/streaming/nextjs/app/api/data/[path]/route.ts (1 hunks)
• templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx (2 hunks)
• templates/types/streaming/nextjs/app/components/ui/chat/widgets/PdfDialog.tsx (1 hunks)
• templates/types/streaming/nextjs/app/components/ui/lib/url.ts (1 hunks)

Files skipped from review as they are similar to previous changes (4)

• templates/types/streaming/nextjs/app/api/data/[path]/route.ts
• templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx
• templates/types/streaming/nextjs/app/components/ui/chat/widgets/PdfDialog.tsx
• templates/types/streaming/nextjs/app/components/ui/lib/url.ts

[coderabbitai]

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between b4b049c and 2cab2d8.

Files selected for processing (1)

• templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx (3 hunks)

Files skipped from review as they are similar to previous changes (1)

• templates/types/streaming/nextjs/app/components/ui/chat/chat-sources.tsx

[marcusschiesser]

great job @thucpn

[coderabbitai]

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 2cab2d8 and 9f9a7ea.

Files selected for processing (1)

• templates/types/streaming/nextjs/package.json (1 hunks)

Files skipped from review as they are similar to previous changes (1)

• templates/types/streaming/nextjs/package.json

[coderabbitai]

Actionable comments posted: 2

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 9f9a7ea and d54fe0a.

Files selected for processing (1)

• templates/types/streaming/fastapi/main.py (3 hunks)

Additional Context Used

Ruff (10)

templates/types/streaming/fastapi/main.py (10)

5-5: Module level import not at top of file

---

6-6: Module level import not at top of file

---

7-7: Module level import not at top of file

---

8-8: Module level import not at top of file

---

9-9: Module level import not at top of file

---

10-10: Module level import not at top of file

---

11-11: Module level import not at top of file

---

12-12: Module level import not at top of file

---

13-13: Module level import not at top of file

---

14-14: Module level import not at top of file

[coderabbitai]

Move the import statement to the top of the file to follow best practices.

- from fastapi.staticfiles import StaticFiles
+ from fastapi.staticfiles import StaticFiles

---

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change

	from fastapi.staticfiles import StaticFiles
	from fastapi.staticfiles import StaticFiles

[coderabbitai]

Tip

Codebase Verification

The data directory is missing or not readable, which will prevent the static file serving configuration from working as intended.

• Ensure the data directory exists in the root of the repository.
• Verify that the directory has the correct read permissions.

Analysis chain

Ensure the data directory exists and has the correct permissions for serving static files.

Scripts Executed

The following scripts were executed for the analysis:

---

Script:

#!/bin/bash
# Description: Verify the existence and permissions of the `data` directory.

# Test: Check if the `data` directory exists and has read permissions.
if [ -d "data" ] && [ -r "data" ]; then
  echo "The 'data' directory exists and is readable."
else
  echo "The 'data' directory is missing or not readable."
  exit 1
fi

Length of output: 139