Off the wall use of Stimulus

I’m just a hobbyist developer but have been using Rails since v 1.0 and stimulus since its introduction.

I tried to update another app from Rails 7.0 to 7.1 and I’m stuck. I have a gem OFX that has never been well maintained. It parses OFX data that is kind of XML, but there are attribute tags that don’t have closing tags. It’s the standard for bank transactions. Rails 7.1 requires a newer version of nokogiri, (which OFX uses) and I don’t think it’s going to get updated.

There are quite a few parsers out there, but it was the only one I could find in Ruby. They all just take an input file and return some kind if Structure.

Looking for a replacement and found ofx-data-extractor which is a Yarn package.

Is there a way to import the package with stimulus and use it in a controller to parse the ofx-data?

Sure you can use ofx-data-extractor in a Stimulus controller
Actually what is advised here GitHub - Fabiopf02/ofx-data-extractor: A module written in TypeScript that provides a utility to extract data from an OFX file in Node.js and Browser in the github readme is exactly how to use it.

If you are using Stimulus you must be familiar with importing some packages into your stimulus controller. For example this is the way to import popper in a Stimulus controller :

i

import { Controller } from "@hotwired/stimulus"
import { createPopper } from '@popperjs/core';

export default class extends Controller {
  
  connect(){   
    }
]

You can do same with ofx-data-extractor

Of course you have to add this package to your node modules folder :
yarn add ofx-data-extractor

The main problem here is that your ofx gem is doing its job backend. But your Stimulus Controller is imported as bulk with your other Stimulus controllers and added to your Rails 7 HTML pages. Such as in application.html
For example this is an extract my app/views/layout/application.html.erb file :

<%= javascript_include_tag "stimulus", "data-turbo-track": "reload", defer: true %>
<%= javascript_include_tag "stimulus_stable_controllers", "data-turbo-track": "reload", defer: true %>

I bundle my Js with ESBUILD, you may have different javascript includes on this page. But if you use Stimulus already you may have some sort of import too in your application.html.erb file.

Now if you want your .ofx files parsed in the backend so you can extract some data and maybe save it in database, you have to do some JS on the backend (rather than the frontend) thanks to the NODE.JS runtine environment.

For example I also use an npm package to do stuff in the backend. Here is the bit of code I use in order to convert a PDF into a DOCX:

class ResumesController < ApplicationController
  def display_docx
  ... stuff here ...
  docx_buffer_base64 = `node ./app/javascript_backend/docx.js "#{html_string}" "#{header_html}" "#{document_options}" "#{footer_html}"`
  ... stuff here ...
  end
end

As you can see I use back ticks: ` which basically means “do something on the machine” or “call a software on my computer and pass arguments to it” (basically subprocess)
You are then a bit outside of your RAILS here…
But what I do is : “call the node.js software on this computer and pass it the file docx.js that is stored inside the rails app and pass it arguments html_string, header_html etc and collect the result”
So basically I do JS inside my Rails app.

If you do this (subprocess) you have to be very careful not to pass user input data as you give access to your computer to potentially harmful code.
But it is working perfectly.

And in my file ./app/javascript_backend/docx.js this is ofcourse not a Stimulus controller but rather plain JS where I still require a library stored in my node moduels folder. In my case I do this :

const HTMLtoDOCX = require('html-to-docx');

const [nodePath, scriptPath, htmlString, headerHTMLString, documentOptions, footerHTMLString] = process.argv;

async function convertToDocx(htmlString, headerHTMLString, documentOptions, footerHTMLString) {

  try {
    const docxBuffer = await HTMLtoDOCX(htmlString, headerHTMLString, documentOptions, footerHTMLString);
  } catch (error) {
    throw error;
  }

}


module.exports = convertToDocx;
convertToDocx(htmlString, headerHTMLString, documentOptions, footerHTMLString)

Then you can get the parsed OFX right inside your Rails controller for you to save in database if required.

To be honest I am not an JS expert and my explantion is probably not faultless but it gives you a general idea. Someone may give you more insight about this…

Also when passing some Js to Node you have to be extra sure your code or the code you are importing don’t contain malware. As per npm page for ofx-data-extractor - npm it is only downloaded 17 times a week which may require a close examination by a keen eye before using.

It feels much safer to raise a ticket on the ofx gem and hope for someone to fix it …

@maxence

Thank you for your suggestions. I actually attempted to import ofx-data-extractor in a stimulus controller and got it to load. Did the yarn add ofx-data-extractor and the controller with only a connect would fire and log a connect.

There was a JS error in the console tied to the ofx-data-extractor that I forget what it said. After I got it to load, I was a little stumped on what to do next.

I went back to the OFX gem to take another look. I remember having a problem a couple years ago. The gentelman who was trying to fix the problem(s) was having problems getting the owner to pull his patch. He sent me a patch version and it fixed the problem. Almost 2 years later they released a new version that I think fixed nokogiri problem. She had a > version and a < version which was about 4 releases back,

I took out my patch version and I didn’t get any errors. Have not tried to upgrade the app to 7.1 yet, but that’s next.

Again thanks

PS Tried to upgrade to rails 7.1 again and OFX still errors out with version of nokogiri is incompatible. There are a few other rails gems that parse OFX I may try them,