Finally Understanding <turbo-stream>

1

I was a bit confused by the <turbo-stream> tag, so much I ended up reading the source code till I figured them out. Here’s what I found.

<turbo-stream? More like <turbo-mutation

The <turbo-stream tag is a web component / custom element that when added to the DOM it will have some jQuery type side effect on the DOM. That’s it!

(Registered as web components here: turbo/src/elements/index.ts)

For example if there’s a div:

<div id="asdf">Before Side Effect</div>

and you add this to the DOM (you can use right-click and inspect element to do this)

<turbo-stream action="replace" target="asdf">
  <template>
    <div id="asdf">LOOK, I HAVE BEEN REPLACED</div>
  </template>
</turbo-stream>

Then it will result in the DOM looking like this (replacing its contents with the template:)

<div id="asdf">
    LOOK, I HAVE BEEN REPLACED
</div>

Does <turbo-stream open HTTP/Websocket connections? Nope.

At first, I thought that adding <turbo-stream to the DOM would open some web socket connection or something. Not so, they don’t do anything but mutate the DOM when they mount. This is done in Turbo by having the browser call connectedCallback which is automatically called whenever a custom element is added to the dom or mutated in any way.

The only “magic” left to understand is how turbo listens to fetch requests and form submissions using it’s StreamObserver. This class will listen to the fetch requests that meet the following criteria:

  • match on the content type of: text/html; turbo-stream (it’s ok if there’s other stuff there in there about UTF8 and such, it just checks that it’s a substring).
  • <turbo-stream> elements are at the root of the request (no wrapping in <body> tags or anything else)
  • <turbo-stream> elements have a <template tag as it’s direct child

When it finds a fetch that matches these criteria then it’ll just add those tags to the DOM and they have the before mentioned side effect(s).

But what about Websockets!?!

Admittedly I haven’t got websockets working yet in phoenix but it seems that websockets are actually just normal fetch requests to the browser, they just get upgraded to a socket. So by listening to fetch requests turbo is also listening to websockets.

All the connecting to a websocket and such has to be handled by your backend framework of choice. This is sadly a bit tricky to do in Phoenix because it doesn’t seem to let you set the headers of a phoenix channel, but with the understanding from above it’s easy to see that it’s not hard to hack around by just listening to a channel and adding the resulting HTML to the DOM.

Conclusion

hopefully that was helpful! I now think of <turbo-stream as a simple way for the server to execute jQuery like commands to the DOM. It’s surprisingly simple and yet works very well. Also I would say don’t be afraid to read Trubo’s code, it’s pretty cleanly written and github was letting me look up where classes were used like this:

image
image1507×588 144 KB

Cheers!

6 Likes
2

I think this is something that I wish the Turbo docs did a better job of pointing out. I believe it’s mentioned that Turbo-streams can be activated by normal HTTP requests, but it seems that regardless there’s still been a lot of confusion about whether streams are websocket-dependent.

3

Yeah, I wish that too. In their defense when you’re using turob-rails a lot of the details of how this works is hidden from you and it does seem like they do those kinds of things (from what I’ve observed).

4

I haven’t made much of an effort to look at how the base Turbo implementation works. Just based on the turbo-rails source I’ve looked at, I’d guess Turbo listens to all submit events and then attempts to fetch a turbo-stream response from the server, correct?

Maybe we could work on a PR to the docs site to make the actual mechanism for responding to normal HTTP requests more clear.

5

Well, turbo sends up an accept header which the server can switch its behavior on. Yeah, it would be cool to make the docs more clear perhaps. Upon more careful reading, it obviously mentions it.

6

So this works for any fetch request? If so that’s pretty cool and I wasn’t aware of that before - I had assumed it was only listening to form submissions and links. I have a drag and drop interface that I’ve been building and upon dropping the element it updates some data using fetch so to have the new element be rendered and returned by the server.

7

Yeah, not sure about this point. Lately, I’ve run into problems trying to get it to work via JS. I resorted to doing a fetch request then adding the results to the DOM (Although this can be a bit awkward if your server returns an error, be mindful of that). I’m thinking that Turbo actually only listened to stuff done by the browser. Tell me, anyone, if I’m wrong.

8

This post was very useful but it still took me a while to understand how to set this up on the server side. Here is the beginner intro to this topic, I am using FastAPI and Python but I’ll try to stay closer to the generic html/http topics.

On the Client

You ship any element to the client with a specific id. This is an element that you plan to mutate with turbo-streams, for instance we have this h3 element:

<h3 id="mutandis">Title to be replaced!</h3>

You also ship a button or a submit form. Why? Well we want to click and get a post response from the server.

<form action="/some_server_endpoint" method="post">
    <button class="button ">Test Streams</button>
</form>

On the Server

The server should accept post requests on /some_server_endpoint url (of course any url will work). Now here is the kick, you want the server to send back response headers like so:

 accept: text/vnd.turbo-stream.html 
 content-length: 137 
 content-type: text/vnd.turbo-stream.html; charset=utf-8 
 date: Mon,25 Jan 2021 16:25:25 GMT 
 server: uvicorn

Two things are critical here:

  • That you set the media_type to text/vnd.turbo-stream.html
  • That you set the header accept to be equal to text/vnd.turbo-stream.html

Here below an implementation on the server side in Python and FastAPI

@router.post("/some_server_endpoint", response_class=HTMLResponse)
async def testing(response: Response, frame_name: str = ""):
    #Build the html turbo-streams here
    return HTMLResponse(
        content= #streams html
        media_type="text/vnd.turbo-stream.html",
        headers={"Accept": "text/vnd.turbo-stream.html"},
    )

The html content of the response should look like this:

<turbo-stream action="replace" id="stream" target="mutandis">
  <template>
    <h3 id="mutandis">Turbo Stream to the rescue!</h3>
  </template>
</turbo-stream>

Note the target mutandis is the same as the id of the h3 element we want to replace in the dom. Turbo will mutate your dom element on the client like so:

<h3 id="mutandis">Turbo Stream to the rescue!</h3>

Now most important of all, why should we do this?

Replacing multiple dom elements with 1 response

We can have these on the client

<h3 id="mutandis1">Title to be replaced!</h3>
<h3 id="mutandis2">Title to be replaced!</h3>

And with just 1 server response like below we can update both elements:

<turbo-stream action="replace" id="stream" target="mutandis1">
  <template>
    <h3 id="mutandis1">Turbo Stream!</h3>
  </template>
</turbo-stream>
<turbo-stream action="replace" id="stream" target="mutandis2">
  <template>
    <h3 id="mutandis2">Turbo Stream!</h3>
  </template>
</turbo-stream>
3 Likes
9

Thank you for posting this info. Like you, I’ve been curious to understand the mechanics a little better. A few things that threw me off, which I think I’ve figured out but need to test some more:

  1. Turbo will only replace turbo-frame elements if the response from the server has content-type text/html. Or at least, it will not do so if the response is text/vnd.turbo-stream.html (FYI, the content type changed from turbo-stream in beta 3).

This was notable for me because I too had noticed you can send multiple <turbo-stream> tags targeting different DOM elements, but you cannot also send any <turbo-frame> elements along in that same response.

Nevertheless, I hope that the muli-turbo-stream thing is a feature and not a bug, as I have already implemented a nodejs middleware that allows for this.

  1. A link does not have be within a <turbo-frame> in order for the response from the server from the link to be able to trigger replacement of DOM elements by a <turbo-stream> in the response. Which is unlike turbo frames, which supposedly only act on links that are children of <turbo-frame> elements.
1 Like