Jamie Gaskins

Ruby/Rails developer, coffee addict

Going back to my first React app

Sep 12, 2015 @ 02:04pm

When I first learned about React, I thought it was such an amazing tool. It made everything so much easier. I still think it's pretty great, but when I went back to the first React app I wrote for work, I realized just how little I understood about using it.

Components stay mounted

One of the most important things I didn't understand was that each DOM node represented by a component has a 1:1 relationship with that component. This means that, as long as that DOM node sticks around (there isn't a node of a different type rendered in its place and it isn't removed entirely), React will use the same component instance for it forever.

This particular app was for ops managers in each market to coordinate delivery schedules with drivers. That's all it did, so the content never changed structure much. This means that the components we used, for the most part, stuck around for the life of the app.

We decided to keep a lot of the data in component state because the app itself was simple. Each block of time on the schedule had some metadata on it, like the number of drivers we needed to be available and the minimum amount the driver would make for being available for deliveries (in case their commissions + tips don't reach that value) for that time segment. Since props are meant to be immutable and we needed to be able to modify some of this data, we simply moved the props to state using getInitialState

The problem is that, when adding a new feature, after days of screwing around trying to figure out why something continued to render stale state, I realized that getInitialState wasn't being called. Turns out, it is called when the component is mounted and then it will never be called again on that component. This makes perfect sense, but is confusing if the lifecycle of a component doesn't match what you think it is. At the time, we thought we'd be getting new components on each render. Somehow, this didn't cause any problems until we began adding new features to it last week.

Once I realized what the problem was, I began iterating on it to make it easier to work with, but I just succeeded in making more of a mess. First, I tried using componentWillReceiveProps to take the new props and update the state — something like this:

ScheduleHeader = React.createClass
  # ...

  getInitialState: -> @props
  componentWilReceiveProps: (nextProps) -> @setState nextProps

But this resulted in consecutive renders (componentWillReceiveProps is caused by a render, then we call setState which starts another one) and it didn't work as well as I thought it would. Then I tried bypassing setState and just using this.state = nextProps inside componentWillReceiveProps. This wasn't any better.

I tried several other equally shortsighted approaches, everything I could think of to make it possible to work with the component in the way it was currently implemented. But it just ended up fixing one bug and causing another. This entire schedule header needed to be gutted. And then, because of the way the header worked with the body of the schedule, it also meant that we needed to do the same there. I was so discouraged I had to go ask Kyle, the one who originally paired with me to write it (we were both learning React together), to help me out.

The good news is that Kyle is one of those developers who never seems to get discouraged by things like this. We ended up rewriting most of the header components (and changing the structure of ones we didn't rewrite) to use state stored in a Redux store, but we got that nearly done in just a few hours.

The lesson here is that, when people recite the React mantra "prefer props over state", this is one of the reasons why. Your component will receive new props on every render, but it might not receive new state because getInitialState will only be called once since it gets reused on the next render.

Component structure can be deceiving

One of the ways we organized our components originally was that each header field determined on its own whether it would render text or a form input to modify its value. I don't have the code in front of me, but it was something like this:

NeededDriverCount = React.createClass
  render: ->
    if @state.editing
      # The EditField component takes all the data needed to perform an AJAX request.
    else if @state.saving

And then I did the same for the other attributes. Each attribute had its own component that did almost the same goddamn thing. So. Much. Duplication.

We ended up refactoring that into a single component that did the same thing for each field, where we could just pass a value and a callback:

ScheduleHeaderHour = React.createClass
  render: ->
    # ...

The EditableField only needs to know the value it's displaying or editing (which one is rendered is based on its own internal state) and a function call when the user presses Enter while editing. That's it! I had originally tried to make the EditField too smart in some ways and not smart enough in others; it constructed its own AJAX request but didn't determine whether it was displaying text or an input. The EditableField component does the opposite: it determines what to display but lets an api object actually update the model on the server.

This refactoring was simple mechanically, but it put everything in the right place conceptually. Sometimes, the way you name things has a lot of influence on how you work with them.

Opal and Transphobia

Jun 18, 2015 @ 09:02pm

Today, Coraline Ada Ehmke, a respected developer in the Ruby community, posted an GitHub issue on the GitHub repository for the Opal project, a Ruby-to-JavaScript compiler. The TL;DR of the issue is "one of the project's core members has posted transphobic tweets; he should be removed."

If you've spoken to me about programming over the past several months, there is a high probability that you know about my love for Opal, stemming from my love for Ruby and my lack thereof for JavaScript, which I also do not try to hide. I wrote the Clearwater web framework with Opal because I love using it that much. I've spoken about it at the B'more on Rails meetup in Baltimore twice now; the first time was in January 2014 about how great Opal is and again in January 2015 when I announced Clearwater. I talked about it at RubyNation just last week. Opal makes front-end development enjoyable for me like nothing else does.

I bring that up because when I tell you that I'm fucking furious at how the Opal team handled Coraline's GitHub issue, I want you to understand all that that implies. Seriously, it's fucking horse shit.

If you read the first response from an Opal core member — whom I know only as "meh" — he dismisses her in about the most insulting way I can think of. He then proceeded to defend his actions throughout the thread.

Admittedly, the title of Coraline's original post toes the line of feeling like dictating who should be on the project's team. However, the body of the post reads more as "hey, you might wanna reevaluate your team". That easily overrides the initial visceral response I had to the title.

I understand the desire to defend yourself in his situation. Whenever someone calls me out for doing something stupid like that (as a cisgender, straight, white man, it's not unheard of for me to overlook my own privilege), it's my first reaction, too. But I also understand how that makes people feel, so rather than act on those defensive impulses, people need to realize maybe they should have a look at what they're doing that might not be in everyone's best interests.

Even if he had posted the dismissive, insulting comment and come back later saying "I'm sorry, I got defensive. I rushed to Elia's defense because he's my friend. I should not have said that.", maybe Coraline wouldn't think any better of him (or the Opal project) — the insult was directed at her, after all — but it might have gone a long way to mitigate the damage he caused in the eyes of the community.

What about Clearwater?

My friend Kurtis expressed interest in Clearwater a while back and I invited him to work on it with me. It's the reason the repository has moved to the clearwater-rb organization. If he had done what meh did today, I would've dropped him from the organization.

Instead, though, he wrote a fantastic article about how Clearwater supports trans developers. I don't want to detract from it so I won't try to TL;DR it here. Please, check it out. It's not long and you've already gotten this far through this one.

A ray of hope

At the time this all went down, I was at the hospital with my mother, so I didn't have much time to contribute to the conversation until well after the damage was done. Once I got home, though, I opened up my laptop to check on the situation and saw this GitHub comment from Adam Beynon, the creator of Opal:

No contributors code is more important than the community at large.

In response to meh and others who are under the impression that a project has no beliefs or feelings toward any particular subject, he had this to say:

Project owners, maintainers and core contributors represent the ethics of a project. They are given those titles for good reason - to represent the project.

This give me some hope here. I was afraid it'd be too little too late, but he's actively pursuing the code of conduct, which will help define what to expect in situations like this moving forward. Adam created Opal; he has full control of the project and has the power to make this right.

Please, Adam. Please make it right.


Thank you to Betsy, who brought this to my attention first. I met Betsy at RubyNation and she was easily one of the most interesting people there (and she wasn't even the one who brought robots). She also spoke up to meh in the GitHub thread.

Thank you to Kurtis for writing that blog post. He wrote it without even talking to me about it. I knew I could trust him completely on issues like this. He is a strong LGBTQ advocate, stronger than I could ever be. His tweets to Elia are a big reason this got so much attention.

Thank you to Coraline, whom I haven't had the chance to meet but who does amazing things for LGBTQ and other marginalized people in tech. Given the great things I've heard from people who have met her, I'm missing out.

Thank you to Nikki Murray, who sent me messages of encouragement through this, letting me know she still thinks my project is great. Considering she likely hadn't ever used Opal before goofing around with Clearwater last weekend, that's huge to me.

Thank you to Adam Beynon for stepping in and doing the right thing — not for his project, but for the members of the community.

And if you're not one of the people I mentioned above, I still appreciate you for reading this all the way through. I know saying that is like the trophy kids get for showing up, but you didn't have to show up here. You chose to. That's important to me.

Clearwater and the Virtual DOM

Apr 06, 2015 @ 09:01pm

I've been experimenting with a lot of things in Clearwater since I first introduced it at B'more on Rails. The reason I showed it to people was because I was hoping to get people talking about front-end development in Ruby with Opal (I've also given a presentation about Opal at B'more on Rails). One of the things that Clearwater does on every link click, at the moment, is rerender the entire app. It does it with a single call, so it's not rendering chunks of HTML here and there, so it's not the worst performance in the world, but I did notice it was clobbering input fields, and micromanaging event handlers (for individually rendered views and attribute-bound blocks) has been a nightmare.

Experiment #1: Components with templates

Components are an interesting way of thinking about web development. Having all of your functionality

One of my experiments has been with adding components to Clearwater. These are Ember-style components rather than React-style, so they still have templates. They're basically a controller and a view in one, and unlike controllers and views, which are instantiated once, the components were designed to be disposable, so you could do something like this in your Slim templates:

  - todos.each do |todo|
    = TodoComponent.new(todo: todo)

But these components have a weird caveat: anything that has events associated with it (basically, anything interactive) needs to let the renderer know about them, so everything from the top of the render tree all the way down has to pass along a reference to the renderer. You had to do this by passing the view's renderer into the component:

    = TodoComponent.new(todo: todo, renderer: renderer)

I've got a few ideas on how to make it better, but for the moment it's just a bit awkward (especially if you're using a bunch of different components in a single view) and it doesn't fix the problem of clobbering the DOM on every link click.

Also, having a lot of small components, each with their own template, means that there's a cost to adding them to your project. Not only do you have to create two files (implementation and template) for each of those components, but each template has to be required into your app, so there's a significant amount of headspace involved.

Experiment #2: Compiling Slim to React

Virtual DOM implementations have become a pretty hot topic lately. First, React shook everything up with their virtual DOM implementation and the idea of just rerendering the entire app any time there's a change. Now, Ember's doing something similar with their new Glimmer engine. I decided to try to use React as the view layer for Clearwater by compiling Slim to React components instead of HTML.

The slim gem depends on a template-compilation gem called temple, which allows you to set up a pipeline where each step in the pipeline takes care of a specific portion of the compilation. What I did was take a Slim::Engine, remove the parts of that pipeline that generate HTML and instead use my own filters to generate React code.

This kinda worked. There were a few challenges and I never quite overcame them. I also realized that React provides a lot more than I need. All I wanted was the virtual DOM. I eventually abandoned this experiment.

Experiment #3: Virtual DOM, no templates

My latest experiment has been using a virtual DOM library, wrapping it with Opal. Initial experiments have been great and I'm currently working on including it into Clearwater.

I realized that this approach would be difficult because I would have the same trouble with converting Slim to virtual-DOM nodes as I did with React, so I decided I'd try it without Slim at all, and just include a DSL for elements:

class TodoList
  include Clearwater::Component

  def initialize(todos)
    @todos = todos

  def render
    div({id: 'todo-list'}, [
      h1(nil, 'Todos'),
      ul({class_name: 'todo-list'},
        @todos.map { |t| Todo.new(t) }

It seems to work really well in my own projects so far, so I've opened a PR on the repo's new home (notice the shiny-new clearwater-rb organization) to encourage conversation about it.

I haven't merged it in yet because it changes how Clearwater renders in a way I haven't found to be backwards-compatible. I'm sure it could be done, but it would require a bunch of work I didn't feel like doing at the time. I wanted to get some feedback from others before I merge it in. Also, I'm not quite done with it.

The virtual-DOM library has a few extra features that I'd like to support, as well, like selective rendering (similar to React's shouldUpdateComponent() function). Clearwater's virtual-DOM support will get there, and once it does, I expect it to be amazing. :-)

Five Stages of Programming Grief

Feb 20, 2015 @ 03:52pm
  1. Denial — There's no way this feature could be broken!
  2. Anger — Ugh! Who wrote this code?!
  3. Bargaining — Alright, I'll work on it if you buy me coffee/pizza/beer
  4. Depression — I can't figure this out. I'm the worst programmer EVER!
  5. Acceptance — Well, it kinda works now. SHIP IT!

Gentle Migration from Marionette to React

Feb 06, 2015 @ 04:48pm

Backbone.js provides a pretty great model layer for front-end web applications, but a lot of the rendering is still DIY. At OrderUp, we've been using Marionette on top of Backbone to handle our rendering in the customer-facing app as well as several internal apps. All was well, we worked out some good usage patterns that kept us productive and the code from becoming spaghettiesque.

My colleague Kyle Fritz developed a new dispatcher to allow us to monitor orders in multiple markets (market = one of the various cities in which we operate) simultaneously. Our original dispatcher could only handle one market at a time, so this was a huge deal for our support team who had to jump around between these markets for every call, email, or customer chat.

To understand what all this entails, I'll go into a few details. First, this app monitors everything in real time, so every time an order comes in or a driver makes themselves available/unavailable for deliveries or updates their location or the status of a delivery ("on my way to pick up the food", "on my way to the customer", etc), we send it to Pusher. The Marionette views in the dispatcher monitored all of these Pusher channels and updated the display whenever any of the underlying models changed.

Spoiler alert: the DOM is slow

Marionette made the development of this new, more powerful dispatcher pretty simple, but it had one major problem: at dinnertime when orders were coming in hot and driver/delivery statuses were getting updated like crazy from hundreds of drivers in multiple markets, all the constant DOM updates were pushing our support team's browsers to the limit.

Kyle and I tried a lot of different tactics to throttle these updates but we couldn't get more than a few percent out of it. We went ahead and opened up a profiler and had a look. Because every Pusher update triggered a DOM update, 80-90% of the time was spent doing this. At peak times, rendering even some of those tiny DOM elements (many of these views were a single <span> tag) was taking longer than we could afford to spend before the next message came in from Pusher.

Maybe Marionette isn't the right tool for this job?

Coincidentally, we had both been learning and playing with React recently. We had already written a new UI for market managers to schedule drivers with it to get our feet wet.

Since React is known for its extremely efficient rendering through laser-focused, coalesced DOM updates, I threw out the idea of rewriting the app with it. Kyle's response was a mix of "that'd be cool" and "I really don't want to rewrite this whole fucking thing."

It then hit me that Marionette views and React components had something in common: their entire operation hinges on the render method. This gave me an idea. What if we replaced some of the bottlenecked Marionette views with React components? Neither of us had any idea if it'd work, so we gave it a shot.

Step 1: Marionette views become React components

One of the biggest bottlenecks the profiler showed was rendering the lists of orders in each market, so we tackled that first.

We took our Order view, did a quick port of its .eco template to .jsx (primarily a matter of converting <%= @foo %> to {@state.foo}), stuffed that in the render method and converted it from Marionette.LayoutView to a React.createClass:

class Views.Order extends Marionette.LayoutView
  tagName: 'tbody'
  className: 'order'
  template: App.JST('orders/templates/order')
    'click': 'showOrderPopover'

  onDestroy: => # ...
  onRender: => # ...

  # etc...

… became …

Order = React.createClass
  render: ->
    # JSX content (what was previously the .eco template) goes here

  componentDidMount: -> # was onRender
  componentWillUnmount: -> # was onDestroy

  handleClick: -> # was showOrderPopover, the click handler in the Marionette view

Notice the onDestroy and onRender callbacks from Marionette had direct React analogs in the form of the componentDidMount and componentWillUnmount methods.

Step 2: Rinse and repeat as high as necessary up the DOM

We did a similar conversion for the Orders view which contains each of the orders. The main difference was that Orders now also mixes in Backbone.React.Component.mixin to help Backbone and React work together. According to the README for this library, this mixin will need to be included in your top-level React component.

Step 3: Make container view render the React components

Then to kick off all of the rendering, we replaced the view that contains this section of the dispatcher with this:

class Views.OrdersList extends Backbone.View
  initialize: (@options) ->

  render: =>
        trigger={@trigger.bind(@)} />
    @ # Backbone views must return `this`.

We used a simple Backbone view because all we cared about was the render method. Once the OrdersList was rendered, it went ahead and rendered the Orders component into its element, which rendered each Order component based on its @collection attribute.

One suspenseful page reload later and, to our amazement, nearly everything worked.

Notice the collection and trigger attributes we passed into Orders. The reason we passed the collection should be obvious if you're familiar with Backbone, but trigger may not be. Some of the functionality of the dispatcher relied on bubbling events up the view hierarchy, so we still needed to call Backbone's trigger method to bubble those events up.

How'd it do?

I'd say it did okay.

Before and after

With Marionette, the Pusher events were happening faster than the DOM updates and the browser couldn't keep up. You can see from that graph that there was almost no idle time. Those events were pegging the CPU so much that even scrolling was impossible sometimes.

We repeated this process on a couple other hot spots in the dispatcher with results that were slightly less spectacular but no less important during our peak hours.

Even on Super Bowl Sunday, easily one of our biggest nights of the year, our new React-based dispatcher handled it like a champ. Given that the Marionette-based one was struggling on an average weeknight, I can't imagine it would've been usable at all that night.

How long did it take?

The majority of the work happened in just a few hours. The React-based dispatcher was deployed to production on the same day we started.

Kyle and I were both amazed at how easily and smoothly we were able to move parts of it from one to the other. We did have a few things to touch up due to event handlers being a bit different between the two, but we didn't spend more than a few days on this total.

So you're saying we should give up on Marionette?

Hell no. Marionette is pretty great at what it does. For most browser apps, the rendering performance is only required to be as fast as the user can navigate the app and, for that, Marionette is just fine. The performance requirement for this specific app wasn't that we had to keep up with a user clicking around the app; it was a data firehose we had to keep up with. React's DOM diffing and coalesced updates could keep up with that.