Roundup December 2020

Roundup December 2020

Welcome in 2021, happy new year!

While the world has seen its share of problems in 2020, it’s been a roller coaster for Hop!

Our community continued to work tirelessly in December, even through the holiday period.

Let’s have a look at what happened in this last burst of productivity for 2020. With this pace of development, 2021 will be a great year for Hop!

A special shout out to Nicolas Adment, who was on an absolute roll in December!

Apache Hop Incubation - 0.50 release preparation

We had two voting rounds for 0.50 release candidates: 0.50-rc1 and 0.50-rc2.

We were told in advance that getting used to “the Apache Way” would take a while. Because of some lingering legacy problems (code, dependency licensing etc), we didn’t reach the required total of 3 positive votes to release 0.50-rc2 as Apache Hop (Incubating) 0.50.

Some of the issues mentioned were already resolved in the code after the 0.50 release branch, and development moved on in preparation for the 0.60 release. We therefore won’t be building a 0.50-rc3 and will move on to 0.60-rc1.

Check the mailing list archives for more information:

UI Updates

Hop Gui continues to evolve. A couple of the changes we made in December:

Single Click UI Actions

Since eliminating right click and double actions improves developer productivity and helps us to move towards a unified user interface on all devices and platforms, we increased focus on single-click actions in the user interface.

New Icons

Nicolas worked hard to update the icons in Hop Gui. A couple of areas where Hop Gui was given some icon love:

• new workflow and pipeline icons
• new toolbar and menu icons
• new and/or updated icons for a number of actions and transforms

The result is a Hop Gui that looks and feels lighter and cleaner:

New Metadata Perspective

A lot of metadata operations (relational, database connections, runtime configurations etc) had their own pop-up configuration dialogs.

These various pop-ups were grouped in one central metadata configuration perpective.

Export to SVG

A neat little feature was introduced to Hop Gui: export to SVG.

Although not spectacular in itself, this little menu item gives you a full svg (scalable) export of your workflow or pipeline graph. For Hop, this is a first step towards self documenting workflows and pipelines.

New Plugins

Two new transform plugins are now available in Hop:

• Transforms:
  • Token Replacement
  • User Defined Java Class

Integration tests

Since nothing works as well as eating your own dog food, Hop now has an integration testing framework that is included in our daily build.

The purpose of the integration testing framework is as simple as useful: code tests can only test a number of cases. Hop was designed and built to be easy. Easy to use, easy to configure, and now easy to test.

Tests are built as workflows and pipelines. The success or failure of a test workflow is determined through the abort or success action exit code.

There is no doubt about the usefulness of this framework: since its introduction (mid-December), the Hop developers have found and fixed a number of bugs that may have been in the code for over a decade!

Partitioned Stream Lookup

The Stream Lookup transform now supports partitioning.

This significantly helps to improve lookups on large volumes of data on large machines.

Docker, Kubernetes updates

The separate Dockerfile and configuration were moved to the main Hop repository.

A Hop Docker image is now one docker pull away through Docker Hub.

Work on Kubernetes continues in the separate repository.

Jandex

All plugins in Hop now use Jandex.

Jandex uses annotations to find and load all plugins at startup. This reduces the startup times for Hop Gui, Hop Run and Hop Server.

Documentation

Lots of existing pages have been extended and/or updated. A selection of the new documentation we have available:

• Integration Testing
• Plugin Development
• Hop SDK
• Creating and checking a release

As always, ping us on mattermost to discuss documentation, and create tickets if you find any missing or incorrect information in the docs. Documentation is code, we take bugs in the documentation just as serious as bugs in the code.

Community

The Hop community continues to grow:

• chat: 122 registered members (up from 108) join
• LinkedIn: 321 followers (up from 309) follow
• Twitter: 250 followers (up from 233) follow
• YouTube: 50 subscribers (up from 40) subscribe
• Facebook: 20 followers (no change) like

The following people have been added as committers:

• Rodrigo Haces (late arrival, not included in November update)
• Jason Chu (late arrival, not included in November update)
• Yannick Mols

Check out the complete list of committers and contributors.

Without community contribution, Hop is just a coding club! Please feel free to join, participate in the discussion, test, file bug tickets on the software or documentation, … Contributing is a lot more than writing code.

Check out our contribution guides to find out more.

Various

Hop Web

Hop Web has seen some impressive activity from Hiromu, target=“blank” over the last months.

The code is in the final stages of being merged into Hop. Once that’s done, you’ll be able to choose to run Hop Gui from a fat (desktop) or web client. Both will be started from the same code base.

Needless to say this will open an entire new world of possible use cases for Hop.

JIRA Tickets

The full list of issues that had activity over the last month is:

Resolved (66)

In Progress (15)

Open (40)

Closed (8)