The Shape of Things

Let’s say you want to build a chair, a simple wooden chair. Nothing fancy, it just needs to support you when you sit and not collapse when you lean back. You want one, or you need one, or your client needs one… whatever.

What’s the first thing you do?

Well, you look at the parts. If you’re smart, you’ll try the instructions first, assuming they exist (they don’t).

So where are the legs and what do they look like? Where is the seat, the backing, the support? Where are the holes and how to they fit? Where are the screws and what are their sizes? How does everything fit together to make this thing I wish to one day sit on?

It all has to do with shapes. Without them, we cannot create anything. Remove the shapes and you remove the parts; remove the parts and you have nothing with which to build.

At this point, you may be tempted to roll your eyes. Please do; I’ll wait. This is really obvious stuff here.

And yet… I continue to watch developers attempt to code large pieces of software without first defining the shapes they want to use in that code.

What happens? Well, they use shapes alright but nobody knows what they are.

"Screw the parts. That’s work and I want to see results. I want pretty things to show up on my screen! Let’s just start with the seat and figure out the rest later as we go! Wheeeee!"

It gets worse when you have more than one developer:

XKCD: Software Development

This isn’t always completely the developer’s fault. Most newer languages today are so type oriented they practically (or actually) require the developer to define types to do anything. But older languages, especially older scripting languages, often have no such requirements and/or if they do, they’re an afterthought.

The biggest offender is, of course, Javascript, in which there are only a few actual types with no true way to create your own.¹ Javascript is a quantum realm where anything can be anything else and everything has one and only one true name: any.

_{¹Yes, there are classes, but that’s just an object with a constructor. Put another way, it’s a function that creates an object that’s no different than any other object: you can add, remove, and change anything on it.}

It is the worst possible language with which to build large scale apps.

Is that just my opinion? Maybe, though I suspect it actually reflects a underlying truth of the universe. Of course, I kind of view all my opinions that way.

Javascript is also the de-facto language of the internet, and so it’s almost unavoidable, but only almost. In truth, Typescript has slowly been… well, not replacing Javascript so much as relegating it to a compiled afterthought. Angular long ago shunted Javascript down into out-of-the-way lib folders in favor of using actual types to create web apps. They never looked back.

React Native tried to half-ass the type system with @flow, but I’ve never actually seen it done. Usually, developers start with flow because it’s Facebook and so is React Native, but they then abandon it somewhere between file eight and file fifty.

React Native can work with Typescript but my first attempt took several days, and even then I ended up with a ton of unsupported modules. It was for this reason that I didn’t like React Native. It may be a perfectly fine platform but Typescript was far too much work to get running and the alternative was Javascript.

So of course I get thrown into a React Native project with hundreds of Javascript files.

I’m being dramatic. Reality was a lot more fun…

Typescript to the rescue

Sometime in the last year or so, Babel (a popular Javascript transpiler) began supporting Typescript. This meant you can have your .ts files sitting right next to your .js files without fear of one knifing the other in the back. More importantly, it means you can progressively refactor your project into Typescript.

So yes, I might have giggled in glee at the prospect. Program in Javascript? No, no thank you. Convert it into Typescript? Yes! Please sir, may I have some more?

This was to me a very unique opportunity. I know why I don’t like Javascript, and I can reason about how it’s a bad language for large projects, but I’ve never actually had to deal with one this size.

It was illuminating to say the least, and for several reasons:

1. It worked. The application wasn’t constantly crashing; the bugs weren’t mutated monstrosities that crawled out of the code to bite you; developers weren’t screaming fire. For the user, this was just like any other app.
2. Nobody actually knew how it worked. When someone needed to fix a bug or add a feature, an inordinate amount of investigation ensued just to figure out what objects were being passed around. Console logs were indispensable. It took me over a month just to get a basic idea of how everything functioned.
3. It was a slow development process. Javascript is a really fast development platform right up to about thirty files, when the project exceeds the cognitive ability of the average developer (maybe even before then). At that point, much of development becomes a process of relearning something you had already known coupled with the confusion that you were the person to write it.
4. Everyone had a different idea about what was going on under the hood. When asked, I got back different explanations for the same code, all of which worked despite being wrong in some aspect or another.
5. Bug fixes often only corrected the technical issue without addressing more systemic problems, mostly because it would take too long. Ex: this function should never be passed undefined, but it was, and we have no idea why, so we’re gonna just add some guard code so it doesn’t crash anymore.
6. There was an incredible amount of guard code. The most popular utility was one called Convert that took in any and spat out what you needed: const number = Convert.toFloat(myValue). These conversions were absolutely essential to keep the app running simply because you could never be sure what you were getting.
7. Entire swaths of the app were no-touch zones. One memorable case involved a single multi-thousand line file that nested functions several levels deep, with objects that were modified on the fly, transformed, and spit out without regard to sanity. This brings me to…
8. Some things could not be "converted" to Typescript; they simply needed to be rewritten. Javascript is an incredibly flexible language that allows you to do pretty much whatever you want, no matter how ill-advised; and some techniques in Javascript simply can’t be used in Typescript.

That last one surprised me. While I had expected difficulties and challenges, I hadn’t expected to be completely stymied by the way the code was written. I had a fairly simple process (more on this below) where I rename the file, figure out and define the interfaces (shapes) of objects being used, then fix the inevitable errors in logic the process had uncovered.

When I came across this toxic monstrosity, I was hooked. The client said not to mess with it with some hand-waving about it needing to be rewritten, but this was a challenge. How could any self-respecting developer back away?

So I spun off a local branch and hacked away in my own time. I gave up several hours later.

Mutation

Javascript has the incredible ability to mutate anything. If I get an object, any object, and I want to add a property, all I have to do is assign the property. If it wasn’t there before, it will be now. If the property was a string, and you assigned a number? Well, now it’s a number. Want an object? That’s fine, just assign away as you skip jubilant through fields of endless possibilities. What could go wrong?

Oh, right. Also, this can be done dynamically.

myObject[serverObject.name] = serverObject.value

Does that look like a dictionary (or map) to you? It certainly does to me, but myObject is a plain old object. What that syntax actually does is add a property to myObject with the name of whatever is returned from serverObject.name. What if myObject already has a property value with that name, one that was defined earlier? It gets replaced.

That bears repeating: it’s possible to write code where json objects returned by the server can dynamically override property values on your objects, or classes, or whatever. And yes, this file was doing exactly that.

What is serverObject.value? Dunno. serverObject is any, but myObject is passed into another function that iterates over something called myObject.types, which is apparently an Array that was added in some other function based on other server values. This function uses the output of that to access another property of myObject using the same kind dictionary-like notation. It then passes that into another function which continue to modify a sub-graph of the original object based on a separate api call, one that’s triggered repeatedly to fill in the object graph while we wait. This all continues through several more functions, all of which add or remove properties from some part or another of the object.

Did you follow that? And what, pray tell, comes out the other end?

Nobody knows. Literally, nobody.

Developers had devolved into console logging the output for a specific server request (because yes, it changed depending on what the server returned) in order to consume the object in the app. And if you think this led to bugs, you would be very much correct, sir. Many bugs. Many many creepy crawlies.

The solution: An extensive and pervasive use of guard code. Just assume anything can be anything else or nothing at all. And thank the gods of code that optional chaining was added to Javascript.

const neededValue = Convert.toString(object?.[object?.types]?.values?.find(o => o.name == this.name));
if (!neededValue) {
    console.log(`This should never happen`);
}

Note: It happened.

So no, some code cannot be converted into Typescript. Some code absolutely needs be rewritten.

However, for most of the code, this was not true.

Regaining Sanity

I’m gonna finish here with the actual process I used to convert the project to Typescript, though it’s far from finished.

1. Start at the bottom; start with the least amount of imports. Importing a .ts file into .js is fine; it doesn’t care since anything goes. But importing a javascript file into Typescript isn’t great. Part of the point of Typescript is to get your code under control, and all .js files are a big fat any. I found it much better to start at the bottom and clean up code with the least amount of .js in it as possible. This allows you to work incrementally and be certain the changes you make actually work properly.
2. Rename the file from .js to .ts. That’s it. I’m assuming, of course, you have Typescript setup and are using Babel.
3. Define your interfaces for all the objects used. You’ll see a ton of errors but most of those will revolve around any. Fix that. Typescript wants types; give them to it.
4. Fix the logic errors. There’s probably gonna be places where the developer assumed a value exists but Typescript says it might not. You’ll find the developer using string when it was in fact a number(that’s a frequent problem) or other type errors. These all need to be fixed after you’ve picked your jaw up and wondered just how the hell this thing worked in the first place (answer: it’s called coercion.)
5. Repeat and Rinse. Find your next file and start the process over.

In most cases, I would start on a Component screen. I’d figure out what individual components and service modules it depends on. I then start with the smallest, easiest files and work my way up.

This depends a lot on your architecture but the general idea is to convert a series of related code objects/components/modules/etc. This will allow Typescript to give the greatest benefit, as it will be checking a logical whole rather than individual parts.

One word of caution: you probably don’t want to start with your most complex files before you’ve converted all other files they depend on. You’ll need the extra help Typescript offers.

Conclusion

I’ve actually enjoyed this conversion process. For one, it brings me great pleasure to shine a light into the darkness, bring order to chaos, and render the inscrutable understandable. It highlights to me that the best code bases aren’t the most clever or the most efficient; the best codebases are the most understandable to the average developer.

To understand our projects, we must know the shape of the pieces we are putting together. We must have a clear mental model of how it all operates. Without those shapes, we’re left in a world of endless possibilities where the name of everything is any… or maybe it’s nothing at all.