Blog

Posts about technology.

Notes on Apache Jena StreamRDFWriter

Notes taken while reading a piece of Apache Jena’s source code.

Apache Jena project is like a box full of interesting things—at least if you love programming. One of its many features, is stream processing.

The graphs in Jena may contain very large datasets, with giga- or terabytes. Some queries may be very large, and then sending the whole result would be simply impracticable.

Instead, the data will go through ARQ. ARQ is a query engine for Jena that supports SPARQL. There is one piece of code there that I found interesting while reviewing a small pull request: org.apache.jena.riot.system.StreamRDFWriter.

It is responsible for writing graph data in a streaming fashion. (See stream processing for programming models and more.)

Stream factories

StreamRDFWriter holds several implementations (as static members) of StreamRDFWriterFactory. The factory has one responsibility only, to create streams (StreamRDF), for a certain format and context.

Using Lumino with Vue

Lumino is the engine that powers the GUI of JupyterLab. It used to be called Phosphor JS, but some time ago there was a misunderstanding and the author gave up maintaining it. Then JupyterLab decided to fork it under this new name.

The documentation is still lacking, but it works as expected, and there is already a community supporting it. We decided to use it in Cylc UI to have tabbed panels like JupyterLab.

This project contains the part of the code of Cylc UI that integrated Lumino and Vue. It was a suggestion from one of the maintainers of JupyterLab, that got involved in an issue of Cylc UI. You can find this project linked too in the Lumino project README file.

Jenkins Active Choices with Jupyter Notebooks

Jenkins Active Choices provides interactive and reactive parameters for the Jenkins UI. It is used by DevOps but also by researchers in industry and academia. Some days ago I ran an experiment where I tried to reproduce similar behaviour of the plug-in with Jupyter Notebooks and ipywidgets.

The experiment was a success, and I am convinced one could achieve the same in Jupyter Notebooks. Jenkins could be used as workflow manager, or the parameters could be submitted to a different workflow manager or another system.

Tweet

You can see more in the following links:

Experimenting with Vuejs and Cytoscape

The project I work on at work had a GUI interface with Python and PyGTK, which is now being ported to the web. We have adopted Vuejs as the JS framework, and have been building the components required for our UI over the past months. The project is hosted on GitHub licensed under GPLv3.

In Cylc 7, an important visualization was the workflow graph. The screen shot below is from the design sketches done by another contributor from the UK, showing how it should look in Cylc 8.

The first library we decided to try is Cytoscape, a well-known graph library, with use cases in research, corporate, and other fields. One limitation of Cytoscape is that the way it renders the graph nodes is limited.

Limited, meaning that you won’t be able to easily add HTML, images, SVG, animation, etc. Though not impossible.

Dynamic breadcrumbs with Vue Router nested routes and Vuetify Breadcrumb component

Vue Router supports nested routes, which allow developers to create a hierarchical navigation structure. This is handy if you want to create breadcrumbs based on this hierarchy dynamically.

Here’s how you should create your nested routes:

const routes = [
  { path: '/', component: { template: `<h1>Home View</h1>` } },
  {
    path: '/users',
    component: {
      render (c) {
        return c('router-view')
      }
    },
    meta: { breadCrumb: 'Users' },
    children: [
      {
        path: '',
        component: {
          template: `
            <div>
              <h1>Users View</h1>
              <router-link to="/users/1">View User 1</router-link>
            </div>
          `
        }
      },
      {
        path: ':id',
        component: {
          render (c) {
            return c('router-view')
          }
        },
        meta: { breadCrumb: 'View User' },
        children: [
          {
            path: '',
            component: {
              template: `
                <div>
                  <h1>User View</h1>
                  <router-link to="/users/1/edit">Edit User</router-link>
                </div>
              `
            }
          },
          {
            path: 'edit',
            component: {
              template: `<h1>Edit User</h1>`
            },
            meta: { breadCrumb: 'Edit User' }
          }
        ]
      }
    ]
  }
]