Turbostream not replacing the DOM

I’d like to replace a modal’s content dynamically, by rendering an empty modal and then use Turbo stream to replace it.

Everything looks OK in my logs and in the browser’s network tab, but somehow the content is not replaced…


Here's how I set it up:
  • Render an initial modal with a Event.new as @event
<%= render partial: 'events/show_event_modal', locals: { event: @event } %>
  • Which contains an element with id="show_event_modal"
<div id='show_event_modal'></div>
  • Plug in events/show_controller.js and bind clicks to displayModal()
<div data-controller="events--show">
  <a data-event-id="some_id" data-action="click->events--show#displayModal"></a>
</div>
  • Fetch EventsController#show
// controllers/events/show_controller.js
displayModal(event) {
  var eventUrl = this.getEventUrl(event);

  fetch(eventUrl)
    .then(this.modal.show)
}
  • Retrieve the corresponding @event & render views/events/show.turbo_stream.erb (the only template present)
# controllers/events_controller.rb
def show
  @event = Event.find(event_params[:id])
end
  • Replace DOM
# views/events/show.turbo_stream.erb
<%= turbo_stream.replace('show_event_modal') do %>
  <%= render partial: 'events/show_modal', locals: { event: @event } %>
<% end %>

Rails logs looks OK:

  Rendering events/show.turbo_stream.erb
  Completed 200 OK

Browser’s Network tab also:

Status  Method  Domain         File                                 Initiator            Type
200     GET     localhost:3000 80b63b5e-dcb1-448c-b4c0-54635667abc1 includes.js:389(xhr) vnd.turbo-stream.html

<turbo-stream action="replace" target="show_event_modal"><template>
  <!-- BEGIN app/views/events/_show_modal.html.erb -->
</template></turbo-stream>

Functionally, the modal is shown empty and the DOM is never replaced.


Could anyone spot the mistake ?


Few things I’ve tried:

  • use only turbo-frames
  • set header 'Accept': 'text/vnd.turbo-stream.html, text/html, application/xhtml+xml'
  • put the content of views/events/show.turbo_stream.erb inside the controller’s action

Could this be a timing thing? If the modal is hidden when the replacement is called, will it be updated? I don’t know what the rules are around turbo-stream and replace. Could you try showing the modal and then replacing after that?

Walter

Could this be a timing thing?

That what I also thought at first (because I were memoizing the modal and I’m not an expert with fetch asynchronicity), but the DOM is never replaced.

If the modal is hidden when the replacement is called, will it be updated?

Good question, I’ll test it right after my answer and update it consequently

Could you try showing the modal and then replacing after that?

Going to test it too !

Thanks for you answer :slight_smile:


edit: I replaced the `displayModal()` function inside the controller by:
displayModal(event) {
  var eventUrl = this.getEventUrl(event);

  this.modal.show();

  fetch(eventUrl, {
    headers: {
      'Accept': 'text/vnd.turbo-stream.html, text/html, application/xhtml+xml'
    }
  })
}

Still no replacement happening :x


edit2: Here’s the corresponding commit, if you think more details are needed: https://github.com/Kawsay/open_calendar/commit/79b810d5fe62e369fc8b0dda00c5df4233730d0b

(App isn’t documented yet, I’ll do what’s needed to make it open-source after v0)

And a development version: https://open-calendar-dev.herokuapp.com/
Credentials are foo@bar.com : foobar, to reproduce the bug: log in, then click on a event

If this is a Bootstrap modal, they have gone back and forth on the expected behavior around modals and their contents. There was a time when you could set a target URL attribute on a modal, and each time it was called forward, it would load that URL. Then for a while, they were cacheing the value of the modal body such that the second time you called it forward, it would hold whatever it used to hold. That led me to write some extra JavaScript around the idea of a reusable modal, that deliberately wiped out that cache each time the modal was closed, so it would be forced to re-fetch.

What you’re doing here, with turbo-streams replace, should not be affected by this at all. But just for grins, try putting a normal visible DIV on screen with that same ID, and get rid of the modal altogether (or just temporarily give it a different ID). That may show you if everything about this replace is happening the way you expect.

Things that can break the replacement:

  • duplicate target IDs in the DOM (more than one possible target for #show_event_modal)
  • Not having the ID show_event_modal in the contents of the template that is returning (I’m not sure if I see that here, but it’s a possible failure point). Make sure that the partial you are rendering actually hard-codes that div id.
  • The replacement not making any sense, structurally, like replacing an LI with a DIV or some other incompatible type (I don’t see that here)

I think you need to attack this on two different fronts. First get the replace to happen at all, then get the replace to happen in the modal where you expect it.

Walter

1 Like

Thanks a lot Walter !

Indeed that’s a Bootstrap modal. What you share is really interesting, having the modal’s body cached should have been quite painful !

Your methodology is truly helping me. None of your 3 points was the error, but that’s still good to know ! I get the replacement to happen on another place, and it made me realize that data-method="get" was required. Adding this attribute to the real link raised an error Form responses must redirect to another location. Doesn’t look directly related, but at least I’m happy to have an error (better than bugs happening silently).

I’ll post an answer when I fixed it !

Watch your console as you click the button, and see what the parameters are (particularly the request type). I wonder if the headers and so forth are what’s stumping it. Because you should be able to just load a regular get request without wrapping it in a fetch.

Walter

I’m about to solve it !

The main issue was that the <a> tag was generated by FullCalendar library, and it doesn’t contain data-method="get" not href. Since I was mapping the link via a data-action and sending the GET inside my Stimulus controller it didn’t work (not sure of the reason though).

The approach I’ve right now is to add the href="/events/<id> to the link, and let the controller display the modal. I’m facing a timing issue, the modal is displayed before the replacement. Only thing I’ve left is to find a way to wait for turbo:frame-render event inside the controller action, and it should be working fine !

I would have prefer handling everything inside the Stimulus controller, but this is fine too. The previous way I’d to handle it was the worst spaghetti you’d ever imagine … :sweat_smile:

Well, not so close.

I’ve spend the last hours fine tuning <a> tags and reproduce HTTP requests with fetch(). Nothing works at this point, but that was interesting to see the different behaviors each combination produces.

Anyway, the main question I’ve now is: does Turbo need an href to perform replacements ?

I’ve reproduced the HTTP request generated by

# it does perform replacement
<%= link_to 'click', event_path(Event.last), method: :get %>

with:

     fetch(eventUrl, {
       method: 'POST',
       body: (`_method=get&authenticity_token=${document.querySelector('meta[name="csrf-token"]').content}`),
       headers: {
         'Accept': 'text/vnd.turbo-stream.html, text/html, application/xhtml+xml',
         'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8',
         'X-CSRF-Token': document.querySelector('meta[name="csrf-token"]').content
       }
     })

Requests are exactly the same (except cookies), but replacement does not happened unless I set an href pointing to /events/<id> on the <a data-action="click->events--show#displayModal"> tag. Doing so does replace the frame, but also creates a timing issue (2 GET requests are sent, 1 by the <a>, another by fetch()). This issue is a reason why I’d like to handle everything in my Stimulus controller.

First thing: links from a tags always default to GET. Rails has the clever turbo-method: :post trick that uses JavaScript to build a “just in time” form to turn a normal GET from a link click into POST (or any other verb).

Second, unless you stop the default link behavior from happening, you will get two requests, just as you said. Since your Stimulus method is listening to the click all you need to do is trap that event so it doesn’t escape your method. (The click on an <a> is a special case, and has a default behavior of issuing a GET request for that link’s href attribute. But an event may have lots of different JavaScript listeners subscribed to it as well. By default, the event travels upward through the DOM tree until it reaches the document [or the window, even], so that every listener that may be interested can “hear” it.)

Copying and pasting your earlier code here as an example:

displayModal(event) {
  var eventUrl = this.getEventUrl(event);

  // stop the default action of a link click
  event.preventDefault();

  // (even more overkill, probably not actually necessary here)
  // this keeps any other listener from hijacking the event and changing it further
  event.stopImmediatePropagation();
  // on with the show...

  fetch(eventUrl)
    .then(this.modal.show)
}

I think this will fix your double-request problem immediately.

Oh, and third, your fetch has a method of POST, but then sends the Rails-specific _method key to set the request back to GET. Very clever. Or very dumb. I can’t tell which, honestly. You may have found a neat way around Hotwire’s insistence on reserving GET requests for turbo-frames.

Walter

1 Like

Thanks a lot Walter, you’re teaching me a lot through this problem (especially with your second point, the way events “travel” the DOM, Rails JIT form injection and .stopImmediatePropagation()). Really, thanks a lot for your time and the knowledge you’re sharing !

About the second point and the code sample you shared, I tested it yesterday but it brings me back to the initial problem: DOM is not replaced, even though I do receive vnd.turbo-stream.html with an id matching a unique element in the DOM, and containing a modal fed with an existing @event.
But yes, indeed, it does stop the double-request problem (and looks like the cleaner way to express it)

Replacement is happening only if:

  • href <a href="/events/<id>> is set
  • data-method <a href=/events/<id> data-method="get">

Because turbo-stream doesn’t replace on the initial GET request (which is the expected behavior of turbo-stream); but it does on the injected POST.

Question is, why no replacement is happening on this POST:

displayModal(event) {
  event.preventDefault()
  var eventUrl = this.getEventUrl(event);

   fetch(eventUrl, {
     method: 'POST',
     body: (`_method=get&authenticity_token=${document.querySelector('meta[name="csrf-token"]').content}`),
     headers: {
       'Accept': 'text/vnd.turbo-stream.html, text/html',
       'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8',
       'X-CSRF-Token': document.querySelector('meta[name="csrf-token"]').content
     }
   }).then(this.modal.show());
}

Should the request appears as a POST server-side ?

I’m about to add a specific route for that and see how it behave

Doesn’t work:

post '/events/:id/update_modal', to: 'events#update_modal'

class EventsController...
  def update_modal
    authorize @event # set to true
    respond_to do |format|
      format.turbo_stream { render :update_modal, status: :accepted }
      format.html {}
    end
  end
end

# views/events/update_modal.turbo_stream.erb
<%= turbo_stream.replace 'show_event_modal' %>
  <%= render partial: 'events/show_modal', locals: { event: @event } %>
<% end %>
// events/show_controller.js
displayModal(event) {
  event.preventDefault()
  var eventUrl = this.getEventUrl(event); // /events/<id>/update_modal

   fetch(eventUrl, {
     method: 'POST',
     body: (`authenticity_token=${document.querySelector('meta[name="csrf-token"]').content}`),
     headers: {
       'Accept': 'text/vnd.turbo-stream.html, text/html',
       'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8',
       'X-CSRF-Token': document.querySelector('meta[name="csrf-token"]').content
     }
   }).then(this.modal.show());
}

Despite:

  • request identified as POST client and server side
  • response containing vnd.turbo-stream.html with valid data
    • Rendered events/update_modal.turbo_stream.erb & 200 OK
  • DOM matching the ID
  • status isn’t 200

I’ve been trying to work through this on my end, because the problem fascinates me. I suspect, but can’t prove, that we are swimming upstream here, trying to use turbo-streams to do what turbo-frames are designed to do. From what I have read so far:

  • Turbo-streams are intended primarily to do wizzy “change it in this browser, see it in that browser” live updates.
  • Your stated goal here is to update a modal for a single person, notably, in the context of a GET request (loading a form into a modal, which is always either a GET or the follow-up render after a failed POST or PUT or PATCH). This seems to be tailor made for how turbo-frames are architected. I may have this totally wrong, but it’s my impression so far.

I’m gonna spend some more time on this problem, because I would really like to understand how to use the New Shiny for this problem that I have always reached to UJS to solve in the past. I deeply understand how to use that technique, since it has been around since we were all using Prototype.js and targeting IE7. I need to get on board with the new hotness.

If you can point me to anything that indicates I have this wrong, I would love to learn more about this. If anyone else is paying attention to this thread, and have any other/better ideas, please pipe up!

Thanks,

Walter

1 Like

Well I’m lucky to have you onboard !

You right about your assumption about my goal.

To go straight to the point, this topic (more precisely this answer) say it all. Turbo intercepts form submissions and click events, and then creates its own request/response cycle. I haven’t finished to read the topic yet, I couldn’t wait for sharing it with you.

I’ll try the dispatchEvent approach, just to try it and see if it’s suitable for this need. It may be an overkill though, using regular XHR + DOM replacement may be a bit cleaner.

1 Like

Here’s what I have working right now. I made a vanilla Rails 7 app with -c bootstrap for the most minimal of possible solutions. GitHub - walterdavis/turbo-example-the-first

The basic structure was a scaffold of a Page, which has a title and a body, and a Block, which has some text. I used the console to make 20 blocks, so I’d have something to play with. I used the scaffolded CRUD UI to make a single page.

I added a method to the PagesController that would select a Block at random and render it through a partial.

That partial in turn renders a turbo-frame tag with the id ‘show_block’:

On the pages/show template, I added these bits to render it.

Now, whenever I click that link (with its target set to match the turbo-frame above) the turbo frame is refreshed with a different random block.

Since these are all GET requests, this all works swimmingly. Now when you wrap a turbo-frame around a form, it hijacks the form submission (PATCH, PUT) to update/replace the frame with the result of the form submission. Since the usual pattern is to re-render a form with errors, and redirect after a successful submission, Turbo paves these cowpaths as well. As long as your form sends a 303 header after a successful submit, Turbo intercepts the request to redirect and re-paints the screen with the result.

I hope this gets you moving again toward your goal!

Walter

1 Like

Quick answer before being off for few days.

I’ve tried your approach: it works like a charm, does the replacement as expected. I think all I’ve to do now is to catch the event (turbo:frame-render I believe) and use event.detail.fetchResponse to identify that the rendering was about the event’s modal.

Great thing is you teached me to use <turbo-frame>, I’m really grateful for this !

I’m back in some days, I’ll post the result when possible.

Thanks again Walter,
Have a great week-end,
Clément

Excellent. Here’s another article/screencast to look at, very thorough, shows how to integrate turbo-streams with turbo-frames, and is focused on precisely the problem you’re tackling: the Bootstrap modal.

Hope this all works out for you!

Walter

Ran into this myself and found out what the original issue was:

when using a fetch request with plain old javascript, you need to make sure you have the appropriate callbacks to allow turbo to utilize the response you get.

ie:

fetch(URL, {
  method: 'POST',
  ...
}.then (response => response.text())
.then(html => Turbo.renderStreamMessage(html));

this was a pain in the ass to figure out but here was a great article that helped me:

2 Likes