4 Years of Developing a Side Project
www.youtube.com
In this video I wanted to share my experience working on an open source side project for nearly 4 years. You can find my project here: https://github.com/kam...

Beginning the Project with Learning Goals

In the summer of 2018, I started a project called Joshuto to learn Rust. At first, I considered learning C++, but became interested in Rust and decided to build something in it instead. While the main goal was for me to learn Rust, over the past four years it has grown into a much larger open source project with many contributors. I wanted to share some of my experiences and lessons learned along the way.

What Joshuto Is and Early Development

I describe Joshuto as a ranger-like terminal file manager written in Rust. For those unfamiliar, ranger is another terminal file manager that makes navigating files and folders in the terminal extremely fast and efficient. The interface usually consists of three columns - the parent directory, the current directory, and a preview of the next directory or file. You can navigate with arrow keys or vim keys, select/cut/copy/paste files, create new tabs and directories, rename files, and more - like a normal GUI file manager. Joshuto also supports bulk file renaming across multiple selected files.

My goal was to learn Rust by challenging myself to build a replacement for ranger that matched all my use cases. I decided to use the ncurses library in Rust since I had experience with ncurses-based terminal programs in the past. I quickly built out a UI to display the current and parent directories, added logic to handle different key presses, custom remappings and themes, and background threads for cut/copy jobs. Under the hood, I used a HashMap to store directory contents instead of a tree structure, which was harder to implement efficiently in Rust.

Reflecting on Design Choices and Major Refactors

Of course, nobody writes perfect code on the first try. Over the years I’ve had to do some major refactors due to early design decisions that didn’t age well.

One of the biggest was switching from ncurses to tui-rs. Ncurses is very primitive without high level abstractions, making reusable code difficult. I could never get things like windows or panels working well, and ncurses can have inconsistent wide character support leading to compile issues for some users. It also caused a lot of screen flickering from refreshing the entire screen. Moving to tui-rs improved the codebase drastically and adoption rate, at the cost of deleting a lot of existing code. This migration took about a week in February 2020 after putting it off for 2 years.

Another refactor was removing the fs-extra library I used for cutting/copying files. Although elegant, it wasn’t efficient - doing unnecessary copies and deletions instead of renames. I wrote a custom implementation to directly rename when possible. There were challenges around handling permissions and edge cases, but it improved performance substantially.

Lessons Learned

Some key lessons I learned:

  • Be pragmatic in your technology choices - don’t force a library or design that isn’t working well. The cost of refactoring later is worth avoiding longer term issues.

  • Write in idiomatic Rust - avoid fighting the language and leverage its strength like enums for event handling.

  • Performance matters more than it may seem at first. Do profiling and optimize bottlenecks.

  • Listen to your users and their pain points to guide development. Their diverse environments and use cases will reveal flaws in your thinking.

  • Open source is extremely rewarding. Seeing others use and contribute to your project is an amazing feeling.

This has been an incredible learning experience. While a ton of work, I’m proud of the project Joshuto has become and still very motivated to keep improving it. I encourage anyone interested in Rust or open source to give it a try!