← Blog

Roundup January 2021

Roundup January 2021

As always at the start of another new month, let’s have a look at what happened at Hop over the last month. Here’s the recap for January 2021!

Apache Hop (Incubating) 0.60 Released

Apache Hop (Incubating) 0.60 was released in late January.

With 0.60 being the first Hop release as an Incubating ASF project, we focused on adopting the “Apache Way”, and made this a source code only release.

Check all details in the release announcement.

Since the release of 0.60, the last license incompatibilities have been resolved: all non-compliant source code has been cleanup up or removed, Hop is now 100% ASF compliant.

Password Encoding

Metadata password encoding has now been made pluggable and is synced with the rest of the Hop platform with the AES, target=“blank” (Advanced Encryption Standard) passwords implementation as a first plugin.

Translation improvements

Having Hop available in one’s native language significantly improves developer productivity. That is why we introduced the Hop Translator tool very early after Hop’s introduction, and have now simplified the process of adding and translating i18n keys for developers and translators.

Developers can now specify i18n keys as i18n:package:key, for example in the annotation below:

@GuiMenuElement(
      root = ID_MAIN_MENU,
      id = ID_MAIN_MENU_FILE,
      label = "i18n::HopGui.Menu.File",
      parentId = ID_MAIN_MENU)

Check the internationalisation documentation, target=“blank” for more details.

On that page, you’ll also find information about the Translator’s use. Check the translation contributor guide to check how you can start contributing translations in your native language.

New Plugins

The list of available plugins in Hop keeps growing. Thanks Matt, Nicolas and Sergio for January’s batch of new plugins:

Kettle/PDI Importer

From the start in late 2019, Hop’s goal was to be an independent platform. Hop started from the Kettle/PDI 8.2.0.7 code base, but both platforms have moved in different directions and are not compatible.

Users and developers who want to migrate their Kettle/PDI projects to Hop are now one File -> Import from Kettle/PDI click away from doing so. Specify your Kettle/PDI jobs and transformations folder, a Hop project or folder to import to, and you’re done. Optionally, kettle.properties, shared.xml and simple-jndi’s jdbc.properties can be imported to Hop variables and relational database connections as well.

Kettle/PDI Import

Check the documentation for more details and don’t forget to create a JIRA ticket if you find any issues.

Various

Continued L&F improvements

We’ve continued to tweak Hop Gui’s look and feel, among other things with improved icons resolution on Mac.

Mac OS Icon

Project improvements

We continue to work on project and environment improvements. After lengthy discussions on the chat ~dev channel and on the mailing list, we made a couple of changes. Projects are now optional, have a default and can inherit from other projects.

Project Improvements

Community

The Hop community continues to grow:

  • chat: 134 registered members (up from 122) join
  • LinkedIn: 375 followers (up from 321) follow
  • Twitter: 291 followers (up from 250) follow
  • YouTube: 68 subscribers (up from 50) subscribe

Check out the complete list of committers and contributors.

Without community contribution, Hop is just a coding club! Please feel free to join, participate in the discussion, test, file bug tickets on the software or documentation, … Contributing is a lot more than writing code.

Check out our contribution guides to find out more.

JIRA Tickets

The full list of issues that had activity over the last month is:

Resolved (71)

Issue Summary Components Created Updated
HOP-1788 Renaming database connection creates duplicates GUI 2020-10-04 2021-01-16
HOP-1855 “Clicking ““get more rows”” closes preview dialog” GUI 2020-10-04 2021-01-16
HOP-1879 find an APL 2.0 alternative for the GPL licensed EDT FTP client in the Put FTP action Actions 2020-10-04 2021-01-27
HOP-2054 Fix the beam-demo project Beam 2020-10-04 2021-01-25
HOP-2262 Native core plugins a registered twice 2020-12-04 2021-01-29
HOP-2308 Enable pritty print on JSON output Transforms 2020-12-16 2021-01-20
HOP-2342 Single click mode: moving a transform or action can cause edit GUI 2020-12-21 2021-01-04
HOP-2376 App icon on macOS looks differently GUI 2021-01-01 2021-01-22
HOP-2378 Translator: implement multi-line code scanner Translations 2021-01-01 2021-01-13
HOP-2381 Add Injection support to JSON Output transform Transforms 2021-01-02 2021-01-27
HOP-2383 Deprecate smart-json and migrate to jackson in JSON transforms Transforms 2021-01-02 2021-01-27
HOP-2384 Add documentation to Enhanced JSON Output Transform Documentation 2021-01-02 2021-01-22
HOP-2385 Replace JFace MessageDialogWithToggle GUI, Hop Web 2021-01-02 2021-01-04
HOP-2387 Make deprecated variables (linesRead, linesWritten,linesOutput.. ) private in BaseTransform 2021-01-02 2021-01-12
HOP-2388 Use environment variable HOP_PLUGIN_BASE_FOLDERS to avoid having to pass it as a system property CLI 2021-01-02 2021-01-09
HOP-2392 Get Fields button not working on JSON Output Transform Transforms 2021-01-03 2021-01-20
HOP-2393 Exporting to SVG should overwrite the file if it exists GUI 2021-01-03 2021-01-03
HOP-2397 “change default of abort transform to ““abort with error”“” Transforms 2021-01-04 2021-01-10
HOP-2398 Remove old Kettle samples, code and welcome pages 2021-01-04 2021-01-04
HOP-2399 add ASF header to adoc/md files Documentation 2021-01-04 2021-01-04
HOP-2401 New design for worklow/pipeline graph GUI 2021-01-04 2021-01-05
HOP-2402 cleanup RAT excludes Build 2021-01-05 2021-01-07
HOP-2403 Context dialog has problems when using the scrollbar 2021-01-05 2021-01-27
HOP-2405 Clean up old transformation and job files 2021-01-06 2021-01-16
HOP-2407 change selected text color Website 2021-01-07 2021-02-02
HOP-2408 add Disclaimer-wip to repo 2021-01-08 2021-01-10
HOP-2409 Update headers to be apache conform 2021-01-08 2021-01-10
HOP-2410 remove edtftpj dependency 2021-01-08 2021-01-27
HOP-2411 no object DCH for MIME type multipart/mixed error sending email in pipeline 2021-01-09 2021-01-16
HOP-2412 hop-assemblies-debug no longer works Build 2021-01-09 2021-01-10
HOP-2414 Update Apache Beam API to 2.27.0 Beam 2021-01-10 2021-01-13
HOP-2418 Split hop does not work GUI, Hop Web 2021-01-11 2021-01-19
HOP-2419 Error display tooltip when transformation failed GUI 2021-01-12 2021-01-12
HOP-2420 Field splitter: add support for escape string Transforms 2021-01-14 2021-01-16
HOP-2421 “”“Open file”” dialog does not show expected files” GUI 2021-01-14 2021-01-16
HOP-2422 Test DB connection does not use variables correctly GUI 2021-01-14 2021-01-18
HOP-2423 Concat fields: doesn’t work with lazily converted data Transforms 2021-01-15 2021-01-16
HOP-2426 Create an action to run pipeline unit tests Actions 2021-01-15 2021-01-16
HOP-2427 Hop icons are not centered when enlarging the icons with ‘Icon size in workspace’ GUI 2021-01-15 2021-01-16
HOP-2429 “When using ““workflow executor”” the metadata is not loaded correctly” Transforms 2021-01-15 2021-01-16
HOP-2430 Concat fields transform throws an error when running inside a container Integration Testing, Transforms 2021-01-16 2021-01-18
HOP-2431 Pipeline errors downstream with table Input using connection variables 2021-01-16 2021-01-18
HOP-2432 The data icon is not positioned correctly GUI 2021-01-16 2021-01-17
HOP-2433 Error creating relational database connection Metadata 2021-01-17 2021-01-18
HOP-2437 JSON Input: Select Fields button doesn’t do anything Transforms 2021-01-18 2021-01-19
HOP-2438 Create action to execute AS400 command Actions 2021-01-18 2021-01-19
HOP-2439 “Rename component ““Web Hop”” to ““Hop Web”” on Jira” Hop Web 2021-01-19 2021-01-19
HOP-2440 “Add ““Metadata structure of stream”” plugin” Transforms 2021-01-19 2021-01-29
HOP-2441 runtime variable not being picked up 2021-01-19 2021-01-19
HOP-2442 Json input still uses system file browser Transforms 2021-01-19 2021-01-27
HOP-2444 Implement global bookmarks for the file dialogs GUI 2021-01-20 2021-01-27
HOP-2445 Removed StyledText from the Hop code GUI 2021-01-20 2021-01-22
HOP-2446 Workflow and Action code cleanup 2021-01-20 2021-01-28
HOP-2447 Reference to wrong doc file in Enhanced JSON Output 2021-01-21 2021-01-27
HOP-2448 Enhanced JSON Output still uses system file dialog Transforms 2021-01-21 2021-01-27
HOP-2449 Add tool to clear database cache GUI 2021-01-22 2021-01-26
HOP-2452 PatternSyntaxException when importing kettle jobs and transformations on Windows Import 2021-01-24 2021-02-02
HOP-2456 Remove annotation property ‘i18nPackage’ 2021-01-25 2021-02-01
HOP-2459 JSON Output still uses file system dialog 2021-01-27 2021-01-27
HOP-2460 XML Output still uses file system browser 2021-01-27 2021-01-27
HOP-2461 XML Input stream still uses file system browser 2021-01-27 2021-01-27
HOP-2462 Get Data from XML still uses file system browser 2021-01-27 2021-01-27
HOP-2464 Remove Sleak and ImageUtil from source code GUI 2021-01-27 2021-01-27
HOP-2465 Create a plugin to encrypt passwords using AES API, Metadata 2021-01-27 2021-01-29
HOP-2470 Remove unused class CertificateGenEncryptUtil API 2021-01-28 2021-01-29
HOP-2472 Some plugins/ folders are scanned twice API 2021-01-28 2021-01-29
HOP-2473 Hop plugin loading is timed but this is never used API 2021-01-28 2021-01-29
HOP-2479 Remove IPluginFolder 2021-01-30 2021-02-01
HOP-2480 fix typos in website Website 2021-01-31 2021-02-02
HOP-2483 Translator does not work under Windows 2021-01-31 2021-02-01

In Progress (7)

Issue Summary Components Created Updated
HOP-1655 In Pipeline and Workflow screens the select action dialog box cannot be moved around the screen or resized. GUI 2020-10-04 2021-01-16
HOP-2104 “Hop needs a default ““Project”” and ““Local Run Configuration”“” 2020-10-04 2021-02-02
HOP-2166 Improve first-time usage experience GUI 2020-11-12 2021-02-01
HOP-2458 See what files are still needed in the client zip to be release compliant 2021-01-26 2021-01-26
HOP-2469 Ensure that Hop complies with ASF encryption policy 2021-01-27 2021-01-28
HOP-2474 Add UDJE (User Defined Java Expression) to ETL Metadata Injection step 2021-01-28 2021-02-01
HOP-2478 Options Dialog : ConfigPlugins are no longer showing up GUI 2021-01-30 2021-02-01

Open (42)

Issue Summary Components Created Updated
HOP-1491 Integrate WebSpoon code Hop Web 2020-10-04 2021-01-16
HOP-1543 Move JDBC properties in BaseDatabaseMeta.class to the database plugins Database 2020-10-04 2021-01-16
HOP-1576 Wrong definition of default Locale in HopGui GUI 2020-10-04 2021-01-16
HOP-1583 Create audit manager 2020-10-04 2021-01-16
HOP-1673 “Replace ““Select Values”” by 3 new Transforms” Transforms 2020-10-04 2021-01-16
HOP-1800 Epic to hold all remaining transforms that have to be ported Transforms 2020-10-04 2021-01-16
HOP-1846 In the Pipeline Run Configuration show which fields are mandatory GUI 2020-10-04 2021-01-16
HOP-1851 Environments with tilde in filename are created in unexpected place on *nix systems GUI 2020-10-04 2021-01-16
HOP-1854 Change of database connection not applied 2020-10-04 2021-01-16
HOP-1857 Data set: Show error if filename or path is wrong 2020-10-04 2021-01-16
HOP-1858 When setting HOP_CONFIG_DIRECTORY, currently no config.json is created 2020-10-04 2021-01-16
HOP-1863 Beam: Error converting Hop data to string lines Beam 2020-10-04 2021-01-16
HOP-1870 “Exception in thread ““Thread-52”” java.lang.RuntimeException: Error starting the Beam pipeline” Beam 2020-10-04 2021-01-16
HOP-1882 Adding run config without name causes exception GUI 2020-10-04 2021-01-16
HOP-1888 Rename package org.apache.hop.pipeline.transform.errorhandling 2020-10-04 2021-01-16
HOP-1903 Keyboard shortcut for Edit Metastore Element GUI 2020-10-04 2021-01-16
HOP-1904 Migrate Generic Connection to Plugin Database 2020-10-04 2021-01-16
HOP-1933 More consistency to hop-conf.sh options/flags CLI 2020-10-04 2021-01-16
HOP-1940 Document the new VFS plugins 2020-10-04 2021-01-16
HOP-1970 Text unreadable in Options dialog GUI 2020-10-04 2021-01-16
HOP-2016 Variable from environment config only work after restart GUI 2020-10-04 2021-01-16
HOP-2027 Fix button positions in dialogs 2020-10-04 2021-01-16
HOP-2037 Remove options from menu when not available for transform GUI 2020-10-04 2021-01-16
HOP-2059 when selecting a folder and not a file an error is thrown 2020-10-04 2021-01-16
HOP-2064 As a data engineer I would like to perform data validation Transforms 2020-10-04 2021-01-27
HOP-2065 Postgres bulk loader leaks connections Transforms 2020-10-04 2021-01-16
HOP-2068 implement metadata injection for all transforms that used the deprecated step api Transforms 2020-10-04 2021-01-16
HOP-2074 Bug seen during find 2020-10-04 2021-01-16
HOP-2076 port sample rows step to transform Transforms 2020-10-04 2021-01-16
HOP-2142 Replace javax/mail/Address dependency Transforms 2020-11-05 2021-01-16
HOP-2434 When reloading project variables they are not refreshed Metadata 2021-01-17 2021-01-23
HOP-2435 Projects: open objects are not remembered in the metadata perspective Metadata, Projects 2021-01-17 2021-01-20
HOP-2450 HopWeb: server crash during preview Hop Web 2021-01-22 2021-01-22
HOP-2451 Hop Web: not possible to see data of a transform Hop Web 2021-01-22 2021-01-22
HOP-2455 attach POM to the correct phase 2021-01-25 2021-01-25
HOP-2463 Provide some samples for Enhanced JSON Output 2021-01-27 2021-01-27
HOP-2468 NPE in file open dialog GUI 2021-01-27 2021-01-27
HOP-2475 Option to build minimal Hop package 2021-01-29 2021-01-29
HOP-2476 Support for Objects and Array data types 2021-01-29 2021-01-29
HOP-2486 “Picking ““Create hop”” eventually brings up action picker again” GUI 2021-02-02 2021-02-02
HOP-2487 Unit tests UI issues 2021-02-02 2021-02-02
HOP-2488 Port MonetDB Bulk Loader to Hop 2021-02-02 2021-02-02

Closed (6)

Issue Summary Components Created Updated
HOP-1862 JSON Output: ERROR: Output fieldname that will contain value is empty! Transforms 2020-10-04 2021-01-16
HOP-2091 port formula step to transform plugin Transforms 2020-10-04 2021-02-02
HOP-2386 Add RAP fragment to the master branch 2021-01-02 2021-01-03
HOP-2389 Remove JFace from hop-assemblies-libs and add it to hop-client GUI, Hop Web 2021-01-03 2021-01-16
HOP-2396 Remove Servlet API jars Hop Server 2021-01-04 2021-01-25
HOP-2466 Detect actions / transforms deprecated by java annotation @Deprecated 2021-01-27 2021-01-28