Word-wise Diff Highlighting

FreshBSD now has support for word lightlighting in Split diffs.

This makes it much easier to notice small changes between two versions of a file, such as fixing a typo or changing a version number.

Effort is made to stop it being too noisy, so not every change will be highlighted, only lines that would otherwise be sufficiently similar. This may need some tuning.

This does break my with_optional_linebreaks helper that prevented wide diffs from making the page extend off too far into the horizontal, but this should be relatively rare.

Fixing that, and adding support for Unified diff view are on the TODO.

A Few Updates

A quick rollup of changes since the new server a few months ago:

Support for HTTP/3

The web server has been advertising HTTP/3 support to clients for months now, but because port 443 was only opened for TCP connections by my firewall configuration, it didn’t actually work.

I’ve rectified this, and will keep and eye on whether it’s actually getting used. There’s currently a roughly 60:40 split between HTTP/1 and HTTP/2 requests, respectively.

Diff line numbers disabled on small displays

Line numbers take up quite a significant chunk of space on mobile devices, so they are now suppressed on displays of less than 800 pixels. If you’re on your phone and need them back, flip it to landscape mode.

Thanks to Mina Galić for requesting this feature.

Titles now include a commit summary

When a single commit is being viewed, the first line of the commit message is now added to the page title to make it more useful for remote link summaries.

Commit IDs are also shortened to the first 7 characters so long Git commit IDs are a bit less overwhelming.

This was also requested by Mina Galić.

You can now link to git commits by shortened hashes. i.e. instead of:

https://freshbsd.org/freebsd/src/commit/1dc1f6bd3138760a9e96e13017cc3c05e5e1b1e9

You can provide a link with just the first 5 or more characters:

https://freshbsd.org/freebsd/src/commit/1dc1f

This is an alias to a prefix search query:

https://freshbsd.org/freebsd/src?q=commit%3A1dc1f*

Which is what you’ll see in the URL bar after the page loads. Perhaps not entirely ideal?

Thanks to Daniel Ebdrup for requesting this improvement.


My contact details are at the bottom of every page, so if you have any requests of your own please do get in touch.

Downtime for Upgrades

You may have noticed a couple of hours downtime today, bringing our current 90 day uptime stat to a shameful 99.854%. But this was for a good cause – working within the venerable guts of the server in order to rip out and replace components long past their prime.

Previously we were powered by a pair of Xeon L5639’s providing 12 Westmere-EP cores running at 2.13GHz, attached to 192GB of memory. They’ve served well, but with their underwhelming performance in the modern era and an idle power of 170W during an eye-watering energy crisis, it’s time for them to join the e-waste/ebay pile.

Their replacement is a single solitary AMD Ryzen 5 5700X – 8 Zen 3 cores at 3.4GHz – with a more modest but hopefully sufficient 64GB of ECC memory. Paired with an Asrock Rack X570D4U motherboard, I also finally have remote management capabilities, woo.

Preliminary tests suggest a very welcome performance uplift of 2-3x across a wide range of workloads, including FreshBSD page generation. Typical front-page loads now take closer to 10ms instead of 20ms, and I measured drilling down to FreeBSD/src taking just 50ms, down from 130ms. Nice, if not exactly life-changing.

More importantly, power – testbench runs measured an idle consumption of just 30W, which is about what I expected. Sadly rehomed in the old Supermicro server case with its redundant hot-swap PSUs, a few more fans and a couple more hard disks, it’s somehow eating 100W. Better than nothing, but clearly more work is needed.

FreshBSD 2021

6 weeks ago I created a branch for a significant rework of FreshBSD. Nearly 300 commits later, and just a week shy of our 15th anniversary, the result is what you’re looking at now. I hope you like it.

There’s loads of internal changes, but here’s the main user-facing stuff:

Improved Look

We have a new vector logo I knocked together in Inkscape, a new drop-down select library, a new pagination widget that offers to reverse the sort order rather than encouraging you to visit page 170,000, and a variety of style tweaks to (hopefully) make things a bit nicer.

Prettier URLs

The site has a new URL scheme, and a much better system for generating them from queries. Where possible they’ll be automatically applied using JavaScript to make sharing links more pleasant. For example, this monstrosity:

/search?q=&project%5B%5D=freebsd&repository%5B%5D=src&branches%5B%5D=releng%2F12.0&sort=commit_date

Becomes:

/freebsd/src/branch/releng/12.0

Old URLs should continue to work.

Use the Source

A new grouped Source filter replaces Project and Repository, making it much easier to go straight to the repositories you’re interested in.

There’s also a new filter for merge commits, though this currently only works for git-based repositories.

More Speed

According to PageSpeed Insights:

  • Old front page: 4,814 DOM elements, 148 KiB, 2.3s to interactive.
  • New front page: 1,616 DOM elements, 37 KiB, 0.5s to interactive

This is a combination of improvements, including:

  • Removal of jQuery and Font-Awesome’s WebFont/CSS
  • Branch/tag/committer filters being suppressed
  • Result counts and aggregations calculated in the background during rendering
  • Multithreaded commit rendering
  • Reduction in per-page results from 50 to 30
  • Improved caching, with a warmup query for the front page on index updates

More to Come

I’ve done a lot of work to improve the quality of the code, which in turn makes further changes easier and more appealing. This also brings us closer to the point at which I’ll be comfortable publishing the codebase.

In the mean time, I hope you continue to find the site useful. If you have any feedback, please email me at tom@hur.st, or Tweet me at @blaagh.

Cheers.

More Pretty URLs, More Speed, Fewer Bugs

A quick summary of some of the user-facing changes since the last news item:

  • Add pretty URLs for repository branches and tags
  • Add /news/yyyy/mm/dd URLs
  • Add a warmup query on indexing to keep the front page fast
  • Add some missing fields to Help
  • Add a source filter combining project and repository
  • Add Cache-Control headers to all pages
  • Replace project and repsitory drop-downs with a source multi-select with groups
  • Replace multiple-select.js with Slim Select for the filter drop-downs
  • Replace FontAwesome’s CSS + WebFont with a few of their SVG icons
  • Remove jQuery
  • Fix empty split diffs on new files
  • Fix filters being forgotten if a search had no results
  • Improve usability on mobile by allowing overflow to scroll rather than clipping

We now score 100 on PageSpeed Insights, with a full front page load taking just 42 KiB. This pairs nicely with the much faster page generation times.

There have been some fairly substantial internal changes as well, but perhaps the most notable is the switch to OpenJDK 16 with a patch to resolve a HotSpot compiler bug FreshBSD ran into about a month ago. Thanks to Charles Oliver Nutter and Aleksey Shipilev for their help.

I’ve also fixed a long-standing resource consumption issue with the commit indexing service, reducing its memory use from ~4-8 GB to under 2 GB. This is filed as RJGit issue #61.

Beta Update

JavaScript is now used to clean up URLs to try to improve their usability. Instead of a mess like:

/?q=foo&project[]=freebsd&repository[]=src&sort=commit_date

You’ll get the much nicer equivalent URL in the location bar:

/freebsd/src?q=foo

The aggregation buttons on these views work better too – previously they only worked on query strings, meaning clicking on one to clear that filter did nothing if you were on a friendly URL. Now they should work everywhere.

Pages should also load a little quicker due to search result totals and aggregation queries running in the background during commit retrieval and rendering.

Lots of other internal changes, but little of it is user-facing.

This past week also saw the first new release of fast_find in nearly six years. This was written for FreshBSD’s CVS importer, and is much quicker than standard Find on JRuby.

New URL Scheme

The beta site now has a new URL scheme in testing.

This replaces the old /project and /commit prefixes with a cleaner style:

  • /project/freebsd/src becomes /freebsd/src
  • /commit/freebsd/src/42 becomes /freebsd/src/commit/42

Old URLs should continue to work as redirects for the forseeable future.

Major Site Update in Beta

If you’re reading this, you’ve found the exciting new beta version of FreshBSD, consisting of a large number of optimizations, bugfixes, and tweaks, with nearly 100 commits and several thousand lines of code changed.

Noteworthy user-facing changes include:

  • Brand new SVG logo and a fresh(ish) new look
  • Significant performance improvements, with a 3x faster front-page
  • Rewritten commit message formatter, which now supports SVN/git revision linking
  • FreeBSD usernames are now included in git committer/author
  • Enabled searching for commiter/authors by partial name
  • This news system, such as it is, is finally hooked up

In addition to these, there have been significant internal code cleanups and improvements which have been long-needed, and the backend PostgreSQL and ElasticSearch have been updated to their latest production releases.

I plan to switch this over to the live site within the next week or so. Please let me know if you spot anything that looks wonky - shoot me an email or poke me on IRC, contact details are in the footer.

FreeBSD Migration to Git

FreeBSD doc and src repositories have migrated from Subversion to Git.

While SVN revision IDs have been preserved, and links to these revisions should continue to work, branches are somewhat confused (git doesn’t actually directly track what branch a commit was made on, you have to painstakingly infer it) and tags are currently missing.

I’m working to resolve these issues, which will also help with other git-using projects such as DragonFlyBSD.