Results

A concrete example of the problem

Thanks to the method I chose when implementing the solution, there's now a concrete example of the problem, which can be used to demonstrate the problem in a practical step-by-step manner.

A practical step-by-step demonstration of the problem follows below.

Imagine you're building the chat application described in Methods: Implementation: Developing an example application, using React.js and TanStack Query.

Initial code

You would probably start out by creating an App component that looks something like this:

src/components/App.tsx

export const App = () => {
    const [
        selectedChatRoomId,
        setSelectedChatRoomId,
    ] = React.useState<string | null>(null);
    return (
        <div>
            <ChatRoomList
                selectedChatRoomId={selectedChatRoomId}
                setSelectedChatRoomId={setSelectedChatRoomId}
            />
            <ChatRoom chatRoomId={selectedChatRoomId} />
        <div/>
    );
};

note

For the sake of brevity only the most relevant source code for this example is shown. For the full source code, see the GitHub repository.

Then you would go on to implement the ChatRoomList component…

src/components/ChatRoomList.tsx

const queryFn = async () => {
    const response = await fetch("/api/chat-rooms");
    return response.json();
};

export const ChatRoomList = ({
    selectedChatRoomId,
    setSelectedChatRoomId
}: ChatRoomListProps) => {
    const {data: chatRooms} = useQuery({
        queryKey: ["chat-rooms"],
        queryFn,
    });
    return (
        <ul>
            {chatRooms?.map((chatRoom) => (
                // Assumed stateless component,
                // implementation not of interest in this example
                <ChatRoomListItem
                    key={chatRoom.id}
                    chatRoom={chatRoom}
                    selected={chatRoom.id === selectedChatRoomId}
                    onClick={() => setSelectedChatRoomId(chatRoom.id)}
                />
            ))}
        <ul/>
    );
};

…and the ChatRoom component.

src/components/ChatRoom.tsx

export const ChatRoom = ({chatRoomId}: ChatRoomProps) => {
    return (
        <div>
            <ChatMessageList chatRoomId={chatRoomId} />
            <ChatMessageInput chatRoomId={chatRoomId} />
        <div/>
    );
};

Now, there are two more components to implement.

The ChatMessageList component…

src/components/ChatMessageList.tsx

const queryFn = async (ctx: QueryFunctionContext<["chat-messages", string]>) => {
    const response = await fetch(`/api/chat-room/${ctx.queryKey[1]}/chat-messages`);
    return response.json();
};

export const ChatMessageList = ({chatRoomId}: ChatMessageListProps) => {
    const {data: chatMessages} = useQuery({
        queryKey: ["chat-messages", chatRoomId],
        queryFn,
    });
    return (
        <ul>
            {chatMessages?.map((chatMessage) => (
                // Assumed stateless component,
                // implementation not of interest in this example
                <ChatMessageListItem
                    key={chatMessage.id}
                    chatMessage={chatMessage}
                />
            ))}
        <ul/>
    );
};

…and the ChatMessageInput component.

src/components/ChatMessageInput.tsx

const mutationFn = async ({
    chatRoomId,
    text,
}: {
    chatRoomId: string;
    text: string;
}) => {
    const response = await fetch(`/api/chat-room/${chatRoomId}/chat-message`, {
        method: "POST",
        headers: {
            "Content-Type": "application/json",
        },
        body: JSON.stringify({text}),
    });
    return response.json();
};

export const ChatMessageInput = ({chatRoomId}: ChatMessageInputProps) => {
    const [text, setText] = React.useState("");
    const {mutate: sendChatMessage, isLoading} = useMutation({
        mutationFn,
        onSuccess: () => setText(""),
    });
    const onChange = (event: React.ChangeEvent<HTMLInputElement>) => {
        setText(event.target.value);
    };
    const onKeyDown = (event: React.KeyboardEvent<HTMLInputElement>) => {
        if (event.key === "Enter") {
            sendChatMessage({chatRoomId, text});
        }
    };
    return (
        <input
            value={text}
            disabled={isLoading}
            onChange={onChange}
            onKeyDown={onKeyDown}
        />
    );
};

Updating UI when new chat messages arrive

You now have a chat application that works to a certain extent but is limited in its functionality in that it doesn't update the UI when new chat messages arrive or when the user sends new chat messages, which is something users have come to expect from modern day chat applications.

Updating the UI when new chat messages arrive can be implemented fairly easily.

In the App component, a useEffect hook can be added that subscribes to a WebSocket connection that receives a message whenever there is a new chat message. Then, using the QueryClient, we can invalidate the queries, causing the queries to be re-fetched and the UI to be updated.

src/components/App.tsx

export const App = () => {
    const queryClient = useQueryClient();
    const [
        selectedChatRoomId,
        setSelectedChatRoomId,
    ] = React.useState<string | null>(null);
    React.useEffect(() => {
        const socket = getSocket();
        socket.on("new-chat-message", () => {
            void queryClient.invalidateQueries();
        })
        return () => socket.disconnect();
    }, [queryClient]);
    return (
        <div>
            <ChatRoomList
                selectedChatRoomId={selectedChatRoomId}
                setSelectedChatRoomId={setSelectedChatRoomId}
            />
            <ChatRoom chatRoomId={selectedChatRoomId} />
        <div/>
    );
};

However, this is very inefficient. Calling QueryClient#invalidateQueries is effectively causing the app to redo all the API requests that it has done so far, which is a lot of unnecessary work.

A better alternative would be to only invalidate the queries that are affected by the new chat message. That is:

• The ["chat-messages"] query for the chat room that the new message arrived in.

• The ["chat-rooms"] query, because the chat rooms are sorted according to when the latest message arrived in the room, and a preview of the latest message is shown in the chat room list item.

Let's assume that the new chat message is included in the socket message, so the information about the chat room that the new message arrived in can be retrieved from the socket message.

socket.on("new-chat-message", (chatMessage: ChatMessage) => {
    void queryClient.invalidateQuery(["chat-messages", chatMessage.chatRoomId]);
    void queryClient.invalidateQuery(["chat-rooms"]);
});

Frankly, this is still inefficient. As the new chat message in included in the socket message, in most cases, all the information necessary is already present in the app's memory in some shape or another, and there is theoretically no need to make any additional API requests.

"In most cases", because no assumption was made that the chat room itself would be included in the socket message, and in case it doesn't exist in cache from before, it would have to be fetched separately before the update can be applied.

QueryClient#setQueryData can be used to manually update the queries' cached data and avoid doing any network requests.

socket.on("new-chat-message", (chatMessage: ChatMessage) => {
    queryClient.setQueryData(
        ["chat-messages", chatMessage.chatRoomId],
        (data) => [
            chatMessage,
            ...(data ?? []), // data might be undefined if the query doesn't exist from before
        ],
    );
    queryClient.setQueryData(["chat-rooms"], (data) => {
        // data might be undefined if the query doesn't exist from before
        if (data == null) {
            return;
        }
        // make a copy of the array which we are allowed to mutate
        const newData = [...data];
        const chatRoomIndex = newData.findIndex(
            (chatRoom) => chatRoom.id === chatMessage.chatRoomId,
        );
        // remove the chat room from the array
        const [chatRoom] = newData.splice(chatRoomIndex, 1);
        // prepend the a clone of chat room object to the array
        // where latestChatMessage is set to the new chat message
        newData.unshift({
            ...chatRoom,
            latestChatMessage: chatMessage,
        });
        return newData;
    });
});

As hinted earlier in the text, this only works in most cases. The case where the chat room doesn't exist in the ["chat-rooms"] query's cached data is not accounted for. It cannot be assumed that the ["chat-rooms"] query's cached data contains all chat rooms. In the example implementation of the ChatRoomList component the ["chat-rooms"] query's cached data will contain the 10 most recently active chat rooms if the query resolved successfully.

In the real implementation which can be found in the GitHub repository and which meets the requirements listed in Methods: Implementation: Developing an example application, "infinite scrolling" has been implemented, which allows the user to view rooms past the 10 first. But even though more than 10 chat rooms may exist in cache, we cannot assume that it's all chat rooms.

There are several ways to address this. Ignoring the case where the chat room doesn't exist in the ["chat-rooms"] query's cached data isn't one of those ways. Even if the new chat message belongs to a room which wasn't among the 10 first rooms, that room is now among the 10 first rooms, since the order of the rooms is determined by when chat messages last arrived in them.

In the example app, reverting to calling queryClient.invalidateQuery(["chat-rooms"]) won't do much harm. Arguably, the added complexity of manually trying to update the cached data isn't worth it in this case.

But for real chat apps that implement pagination, the situation is arguably different. It might then be a question of potentially re-fetching hundreds of chat rooms just because one chat room got a new message.

Handling the case where the chat room doesn't exist in the ["chat-rooms"] query's cached data

The easiest solution from the point of view of a frontend developer is probably if the backend makes a change to include the chat room object as well as the chat message object in the socket message which informs the client about the new chat message.

But the socket service adapting this way to accommodate to very specific frontend needs is a luxury which cannot be expected every time a case like this is encountered. The socket service has performance and efficiency considerations of its own that it needs to care about, and bundling more data in the socket messages goes strictly against those considerations.

So, as a frontend developer, you might as well accept that you need to be able to handle this in the frontend.

The first step is to go asynchronous.

src/socket-message-handler/onNewChatMessage.ts

export const onNewChatMessage = async (
    queryClient: QueryClient,
    chatMessage: ChatMessage,
) => {
    queryClient.setQueryData(
        ["chat-messages", chatMessage.chatRoomId],
        (data) => [
            chatMessage,
            ...(data ?? []), // data might be undefined if the query doesn't exist from before
        ],
    );
    const data = queryClient.getQueryData(["chat-rooms"]);
    // make a copy of the array which we are allowed to mutate
    const newData = [...(data ?? [])];
    const chatRoomIndex = newData.findIndex(
        (chatRoom) => chatRoom.id === chatMessage.chatRoomId,
    );
    let chatRoom: ChatRoom;
    // if chat room doesn't exist, fetch it
    if (chatRoomIndex === -1) {
        try {
            chatRoom = await fetchChatRoom(chatMessage.chatRoomId);
        } catch {
            // if fetching the chat room fails, we surrender (for now)
            return;
        }
    }
    // else remove it from the array
    else {
        const staleChatRoom = newData.splice(chatRoomIndex, 1)[0];
        chatRoom = {
            ...staleChatRoom,
            latestChatMessage: chatMessage,
        };
    }
    // prepend chat room to the array
    newData.unshift(chatRoom);
    queryClient.setQueryData(["chat-rooms"], newData);
};

socket.on("new-chat-message", (chatMessage: ChatMessage) => {
    queryClient.setQueryData(
        ["chat-messages", chatMessage.chatRoomId],
        ...
    );
    queryClient.setQueryData(["chat-rooms"], (data) => {
        ...
    });
    void onNewChatMessage(queryClient, chatMessage);
});

Dealing with concurrency

The onNewChatMessage function does not take into consideration if any of the queries whose cached data it's modifying are currently in-flight. This could affect the outcome in different unwanted ways.

To dodge this potential bullet, the query which the function is currently operating on must be cancelled if it's currently in-flight. It can be done by using QueryClient#cancelQueries. Then after doing the synchronous query cache update, if the query was being fetched prior to the update, the function must call QueryClient#refetchQueries to make sure that whatever was being fetched prior to the update still gets fetched in the end.

src/socket-message-handler/onNewChatMessage.ts

export const onNewChatMessage = async (
    queryClient: QueryClient,
    chatMessage: ChatMessage,
) => {
    const wasFetchingChatMessages =
        queryClient.isFetching(["chat-messages", chatMessage.chatRoomId]) > 0;
    if (wasFetchingChatMessages) {
        await queryClient.cancelQueries([
            "chat-messages",
            chatMessage.chatRoomId,
        ]);
    }
    queryClient.setQueryData(
        ["chat-messages", chatMessage.chatRoomId],
        (data) => [
            chatMessage,
            ...(data ?? []), // data might be undefined if the query doesn't exist from before
        ],
    );
    if (wasFetchingChatMessages) {
        void queryClient.refetchQueries([
            "chat-messages",
            chatMessage.chatRoomId,
        ]);
    }
    const wasFetchingChatRooms = queryClient.isFetching(["chat-rooms"]) > 0;
    if (wasFetchingChatRooms) {
        await queryClient.cancelQueries(["chat-rooms"]);
    }
    const data = queryClient.getQueryData(["chat-rooms"]);
    // make a copy of the array which we are allowed to mutate
    const newData = [...(data ?? [])];
    const chatRoomIndex = newData.findIndex(
        (chatRoom) => chatRoom.id === chatMessage.chatRoomId,
    );
    let chatRoom: ChatRoom;
    // if chat room doesn't exist, fetch it
    if (chatRoomIndex === -1) {
        try {
            chatRoom = await fetchChatRoom(chatMessage.chatRoomId);
        } catch {
            // if fetching the chat room fails, we surrender (for now)
            return;
        }
    }
    // else remove it from the array
    else {
        const staleChatRoom = newData.splice(chatRoomIndex, 1)[0];
        chatRoom = {
            ...staleChatRoom,
            latestChatMessage: chatMessage,
        };
    }
    // prepend chat room to the array
    newData.unshift(chatRoom);
    queryClient.setQueryData(["chat-rooms"], newData);
    if (wasFetchingChatMessages) {
        void queryClient.refetchQueries(["chat-rooms"]);
    }
};

Since fetchChatRoom is asynchronous, by the time it's finished, the cached data for the ["chat-rooms"] query might already have changed and differ from what's in newData.

src/socket-message-handler/onNewChatMessage.ts

export const onNewChatMessage = async (
    queryClient: QueryClient,
    chatMessage: ChatMessage,
) => {
    const wasFetchingChatMessages =
        queryClient.isFetching(["chat-messages", chatMessage.chatRoomId]) > 0;
    if (wasFetchingChatMessages) {
        await queryClient.cancelQueries([
            "chat-messages",
            chatMessage.chatRoomId,
        ]);
    }
    queryClient.setQueryData(
        ["chat-messages", chatMessage.chatRoomId],
        (data) => [
            chatMessage,
            ...(data ?? []), // data might be undefined if the query doesn't exist from before
        ],
    );
    if (wasFetchingChatMessages) {
        void queryClient.refetchQueries([
            "chat-messages",
            chatMessage.chatRoomId,
        ]);
    }
    let wasFetchingChatRooms = queryClient.isFetching(["chat-rooms"]) > 0;
    if (wasFetchingChatRooms) {
        await queryClient.cancelQueries(["chat-rooms"]);
    }
    const data = queryClient.getQueryData(["chat-rooms"]);
    // make a copy of the array which we are allowed to mutate
    const newData = [...(data ?? [])];
    const chatRoomIndex = newData.findIndex(
        (chatRoom) => chatRoom.id === chatMessage.chatRoomId,
    );
    let data = queryClient.getQueryData(["chat-rooms"]);
    // make a copy of the array which we are allowed to mutate
    let newData = [...(data ?? [])];
    let chatRoomIndex = newData.findIndex(
        (chatRoom) => chatRoom.id === chatMessage.chatRoomId,
    );
    let chatRoom: ChatRoom;
    // if chat room doesn't exist, fetch it
    if (chatRoomIndex === -1) {
        try {
            // synchronous handling of socket message ends here
            chatRoom = await fetchChatRoom(chatMessage.chatRoomId);
        } catch {
            // if fetching the chat room fails, we surrender (for now)
            return;
        }
        // we need to check for in-flight queries and cancel them again
        // since we "left" the synchronous execution context
        wasFetchingChatRooms = queryClient.isFetching(["chat-rooms"]) > 0;
        if (wasFetchingChatRooms) {
            await queryClient.cancelQueries(["chat-rooms"]);
        }
        data = queryClient.getQueryData(["chat-rooms"]);
        newData = [...(data ?? [])];
        chatRoomIndex = newData.findIndex(
            (chatRoom) => chatRoom.id === chatMessage.chatRoomId,
        );
        if (chatRoomIndex !== -1) {
            // we can assume that the chat room we just fetched
            // is as up-to-date or more up-to-date than the one
            // that was added to the cache while we were fetching
            newData[chatRoomIndex] = chatRoom;
        } else {
            // prepend chat room to the array
            newData.unshift(chatRoom);
        }
        // we cannot assume that the chat room we just fetched
        // should still go to the top of the list
        newData.sort(latestChatMessageCreatedAtDescendingCompareFn);
    }
    // else remove it from the array
    else {
        const staleChatRoom = newData.splice(chatRoomIndex, 1)[0];
        chatRoom = {
            ...staleChatRoom,
            latestChatMessage: chatMessage,
        };
        // prepend chat room to the array
        newData.unshift(chatRoom);
    }
    // prepend chat room to the array
    newData.unshift(chatRoom);
    queryClient.setQueryData(["chat-rooms"], newData);
    if (wasFetchingChatMessages) {
        void queryClient.refetchQueries(["chat-rooms"]);
    }
};

Now that concurrency has been taken into consideration, this particular query cache updater is finished.

Summing it up

The onNewChatMessage function was just one query cache updater. As the app grows, the need to create more of these updaters will arise, and while the code can be organized into neat file structures and parts of it can be extracted into helper functions that can be reused across those files, there will still no doubt be a lot of query cache updating code to maintain.

A single forgotten conditional statement or unhandled edge-case in any of these query cache updaters can lead to a cascade of bugs that are hard to track down and fix.

Whenever there are changes to the data model or the UI of the application, it's likely that some changes need to be made to the query cache updaters as well, since they are tightly coupled with both the data model and the UI of the application.

This code is not something you should have to write and maintain yourself, especially since there are other solutions (such as Orbit.js and the Cloud Firestore client-side SDK) that prove that the desired functionality can be achieved in other ways which eliminate the need for you to write this kind of code and — in addition to that — work more reliably.

The solution

The solution comprises of two libraries:

• @tanstack-query-with-orbitjs/core

• @tanstack-query-with-orbitjs/react

@tanstack-query-with-orbitjs/core

@tanstack-query-with-orbitjs/core is effectively an extension of, a wrapper of, a flavor of or a preset for @tanstack/query-core; the UI framework agnostic core of TanStack Query.

@tanstack-query-with-orbitjs/core is also UI framework agnostic. As the name suggests, it's a library for using TanStack Query together with Orbit.js. It exports the following items:

• LiveQueryClient
  Replacement for QueryClient. (Extends QueryClient class.)

• LiveQueryClientConfig
  Type definition. (Extends QueryClientConfig interface.)

• QueryMeta
  Module augmented, and declaration merged more specific version of the QueryMeta interface.

• GetQueryOrExpressions
  Type definition for a function signature which has a central role when using the library.

• LiveQueryObserver
  Replacement for QueryObserver. (Extends QueryObserver class.)

• LiveInfiniteQueryObserver
  Replacement for InfiniteQueryObserver. (Extends InfiniteQueryObserver class.)

Using @tanstack-query-with-orbitjs/core differs from using @tanstack/query-core in that the LiveQueryClient constructor requires that you pass it a reference to an Orbit.js MemorySource and in that you don't pass queryFns to observers when instantiating them. Instead, the library makes use of the meta object that @tanstack/query-core associates with each query. In the meta object, you specify a getQueryOrExpressions function which returns the query or expression which the default queryFn uses to query the memory source and which is used to create an Orbit.js live query which automatically keeps your query up to date.

For more information, check out the library's README.md, where how to use the library is covered in greater detail, or check out the source code for the example chat application which uses this solution.

@tanstack-query-with-orbitjs/react

@tanstack-query-with-orbitjs/react contains React bindings for using @tanstack-query-with-orbitjs/core in a React application. It exports:

• useLiveQuery
  Same as useQuery, but for when using @tanstack-query-with-orbitjs/core.

• useLiveInfiniteQuery
  Same as useInfiniteQuery, but for when using @tanstack-query-with-orbitjs/core.

• useLiveQueryClient
  Same as useQueryClient, but for when using @tanstack-query-with-orbitjs/core.

For more information, check out the library's README.md, where how to use the library is covered in greater detail, or check out the source code for the example chat application which uses this solution.