Citations in custom RAG pipeline

[ClancyD-SHC]

Can we pass through RAG metadata to show as chunks like with the local RAG interface from a pipeline?

Any help appreciated!

[csvance]

The biggest problem with OpenWebUI / Pipelines for custom RAG applications is not supporting citations. There are cases even with 400B+ parameter models where the LLM does not correctly synthesize the information from the retrieved documents or understand that it does not have sufficient context to answer. In these cases, the user has no easy way to verify the information since pipelines can only return a string (or character generator for streaming output).

RAG without citations is dangerous. I'm open to trying to contribute something here, but the issue is with current pipeline abstractions there is no way to handle this.

The issue with including all of the context in the response string is this will quickly saturate the context length for the chat history. Ideally the history shouldn't contain all of the context information for every previous response, it should only contain the users messages and the agents response. Even if the context was included in a way that was filtered out of the history, its still not a great way to present the information to the user.

A potential workaround creating an HTTP endpoint on the custom RAG service to display all of the context, and including a link to it in a way that would be parsed out for the chat history, like [<citations>](https://custom-rag-service/citations/123) - This is still kind of ugly and requires extra work, but that would also add potential for user provided feedback for the reranking model which is something that even OpenWebUI native RAG support lacks.

Another potential option is defining a standard XML/JSON citation schema for use at the end of the response which OpenWebUI could parse into citations. The disadvantage of this is it only works with simple pipeline structures. Both sides of the pipeline would have to be aware of the format, which would make any kind of composition or scaffolding difficult. To me most of the power of pipelines to begin with is the ability to easily connect an entire application into OpenWebUI's interface. Only a single pipeline is needed for such cases realistically because all of the heavy lifting will happen in the applications code rather than OpenWebUI's pipeline.

[etiennebonnafoux]

There is also #229

[Maleya]

Any news or developments on this topic?

[codefox1387]

For Pipes not Pipelines...If you decompose the existing RAG codebase in OWUI you will find that it leverages an event emitter with the type "citation." If you use this event emitter in your tool, function, pipeline you can pass data to it to generate citations in the standard OWUI format with the chips at the bottom of the response.

This won't help for pipelines though.

You will need a method to get those citations from whatever RAG service you are using to pass them to the emitter and not use them in the response body.

Here is a snippet of an example leveraging the BBC news feed example code. This example runs for every article found/used in the search and generates a citation with the source link and the content from the article:

Emit citation for this article

            await __event_emitter__(
                {
                    "type": "citation",
                    "data": {
                        "document": [article["content"]],
                        "metadata": [{"source": article["link"]}],
                        "source": {"name": article["title"]},
                    },
                }
            )

[Maleya]

Hello! As far as I have understood, pipelines do not support event_emitters (see #225). Have you found a way around this or does this only apply for functions?

[codefox1387]

Ah you're right, pipe/pipes support OWUI internal functions but not pipelines. Sorry the terminology is a bit confusing, I edited my response.

[marchellodev]

While not ideal, I am outputting custom messages (tool calling, citations) like so:

<details>
<summary>Citations</summary>
- Citation 1
- Citation 2
</details>

[marchellodev]

Having said that, you then have to manually pre-process messages to remove this kind of "UI" -- just to not confuse the LLM

Using __event_emitter__ through SSE would be much better

[pfedasullivan]

I've been able to get native citation rendering working with this method in our rag pipeline:
`

   async def outlet(self, body: dict, user: dict) -> dict:
    body["messages"][-1]["sources"] = self.build_citations()
    return body`

build_citations just returns an array of dictionary objects with the document, metadata, and source values like people have mentioned above.

[rhajou]

@Maleya I found that you can pass url as well in source, and you can aggregated mulitple documents in the list provided to "document" it helps aggregate citations in the GUI

[Maleya]

I noticed that as well! Documents with the same source are aggregated under the same reference 'pill'. Thanks though @rhajou, that one is easy to miss.

[rhajou]

@Maleya but still I didn't understand the relationship or role of metadata (source inside) and "source" key . For the time being I am keeping both with same value... Would love a clear full documentation on citations from the Open Webui team.

[ericsua]

@LuyiTian @Maleya @rhajou Sorry for the OT, but does anyone know if Pipelines can also return the message's stats as in the models directly handled by Open WebUI (see image below) in the metadata or in another field of the body with an example, or if there is a way to tell Open WebUI to compute them? Thanks ☺️

[pfedasullivan]

@LuyiTian Oh cool. Thanks for sharing, and I learned something new. I was able to add distances using code like @Maleya mentions.