Cloje Status Update (Taking Time Off)

What's been going on with Cloje lately? Here's the short version:

• I tried to add immutable types to Cloje. That turned out to be a bad decision at this stage of the project. So I'm backing off of that.
• I'm taking a month off from Cloje to recharge and reflect.

Immutable data structures

As you can tell from previous posts (1, 2), I did a lot of thinking and planning about how to add immutable data structures to Cloje. (Perhaps too much thinking and planning.)

Immutable data structures are a very important part of Clojure, and a really good idea in general, so I really want to have them in Cloje. And I know that the longer I wait to add them, the more disruptive a change it will be, both in terms of impact on users, and implementation effort required (because there will be more functions in Cloje to update).

So, I wanted to at least add placeholder types, to get the internal and external APIs nailed down as soon as possible. The placeholder types would just be wrappers around existing, mutable types. Users can't touch the underlying data, so they are effectively immutable. Then, we can come back later to make the implementation more efficient, without disrupting any functionality.

It was a decent plan. Do just enough planning and work now, so that the path is clear and smooth for the future. Unfortunately, I underestimated how much of a slog it would be.

There are four types that I had plans to create placeholder types for: vectors, hash maps, lists, and strings. I decided to start with strings first, because although there were a lot of functions to update, most of them were simple and isolated from the others, so I could just knock them out one by one, and the mini-accomplishments would hopefully keep my momentum and motivation up.

Implementing the placeholder types themselves is easy. They're just records (structures) with a slot to hold a host-type value, and a handful of functions for safely working with them. It's also pretty straightforward to update most functions. Just convert or extract the internal value, perform the existing operation on it, and wrap the result.

The slog comes from three things:

• Updating the unit tests for all the affected functions. (Updating the tests has generally taken 10 to 20 times longer than updating the functions themselves, mainly due to refactoring tests to avoid unmanageable levels of duplication.)
• Untangling dependencies between related functions. (E.g. function X uses function Y internally, so first function X has to be updated to accommodate changes in function Y's behavior, then function Y can be updated, then function X can be updated the rest of the way.)
• Gradually uncovering more and more functions that need to be updated. (I initially estimated that about 25 functions would need to be updated for strings. The list now has over 40 functions, about 30 of them finished. And there are probably another 30 functions that don't need to be updated, but do need new tests.)

I was initially anticipating each type to take 1-2 weeks, so my hope was to get at least strings and lists done for Cloje 0.3, and still have time/energy for some small features, maintenance, and project management. But strings by themselves took over 3 weeks before I stopped, and there's still more work to do. And it drained a lot of my motivation to work on the project, which interfered with project management and recruitment. That one "feature" effectively consumed an entire release cycle, and it's still not done yet, so there's nothing to ship. Ugh.

In hindsight, attempting immutable data structures at this stage in the project was a bad decision. They are not fun to work on, and there's no big pay-off when finished. It was also a bad idea to attempt them while still being the only person on the project, since there was no one to help shoulder the burden, nor even to commiserate with. Plus it's not an exciting feature that would help attract contributors or users, so it doesn't push the project forward.

So, even though I know that postponing it will mean more total work to implement immutable data structures later, it's just not practical to do it now. The work I put into immutable strings will be left in a branch, but I'm putting it and the other immutable types aside, at least for now.

Taking some time off

Cloje started as an experiment, to see how closely Clojure could be cloned on top of Scheme, and just as importantly, to have fun and learn by doing it.

It has now been about 6 months, and the results of the experiment have been mixed. I have learned a lot, and had some fun, and successfully cloned many superficial Clojure features. But there has also been a lot of frustration, disappointment, and lowering my sights regarding cross-cutting features and general compatibility with Clojure.

I have had a lot of questions about this project on my mind lately:

• Does the idea of Cloje as an integrated clone of Clojure make sense anymore? Does it provide any benefit? Or would it make more sense to create many separate libraries, each focused on providing one specific feature?
• Is this project worth investing more time and energy into? Is it worth trying to attract more contributors? Is creating a janky almost-clone of Clojure a worthwhile use of anyone's time?
• Can the project provide other value to make it more worthwhile, for example as a learning environment or incubator for technologists? What is the best way to do that? Is Cloje a good vehicle for that?
• How should the project attract more contributors? What documentation is needed in order to be ready for incoming contributors? What steps can be taken to make a diverse and inclusive team?
• If the project attracts more contributors, how should authority and responsibility be divided? How much organizational structure should be established now, and how much should wait until more people are already involved?
• What technical infrastructure does the project need? Will GitLab be enough, or will we need a mailing list, IRC channel, and so on?
• Should Cloje be relicensed to a different FLOSS license? If so, which license? Would having a different license than Clojure expose the project to additional legal risk?
• How should copyright on contributions be handled? Does the project need a contributor license agreement? How rigorous would it need to be? Would it discourage people from contributing?
• Is it possible for Cloje to exist without becoming entangled in technological rivalries (e.g. Common Lisp vs. Scheme vs. Clojure vs. Java)? Or will it inevitably act as a focal point for bickering and snobbery? (Cloje recently got noticed by a few people, and shared on some social networks and programmer forums. Cloje itself got generally favorable responses, but even the positive responses were usually laced with rivalry or belittling of one language or another. It reminded me how toxic programmer culture can be.)

Right now, my instinct is to say, "Well, it was an interesting experiment, but it's not worth continuing. Best to just wrap it up and move on." But there's a chance that's just burnout talking. So, I'm going to take a month off from the project to recharge and reflect, then revisit these questions in late September.