Posts about technology and arts.
A few days ago I saw a post at the Cylc Google Group, about file permissions for files generated by Cylc. The post was related to content created by Cylc, but in an environment with PBS.
For context, Cylc is an Open Source meta-scheduler, written in Python, that allows you to define cycle points with dependencies. These cycle points can be simple incremental integer numbers, or ISO8601 periods or points (e.g. run every 5 minutes, from 10 days ago until the next year). Cylc takes care to create an execution schedule for you, and delegate that to a system that runs your workflow. I work full time on this amazing Open Source tool!
Such system could be the local computer in background, batch systems such as
at, or PBS.
PBS was created for NASA, to manage executing jobs taking into consideration cluster resources,
and also using queues, priorities, and other features useful for HPC programming. Later PBS
was acquired by Altair, an Open Source version OpenPBS was created, and later abandoned. And
there is another fork called PBS Torque. I first encountered PBS at the São Paulo
University, in Brazil, where they had a PBS Torque cluster.
Even though I have access to an environment with Cylc and with PBS, I decided to give it a try and see how hard it would be to reproduce it with Docker. One thing that I like about this approach is the possibility to share the work with others online. I believe it improves communication, agility, and can be useful for posterity.
I grew up watching the weather forecast in Sao Paulo, Brazil. Where it is called “previsão do tempo”, which is the literal translation in Portuguese.
This post contains some recent screen shots of the weather forecast as it is presented in one of the main TV channels, as well as some screenshots from an online video of other local institutions that either provide the forecast, or help running the NWP models.
Started on a new project last Monday. One of the tasks in this project involves a new design for the Web layer. And as the application is quite similar to JupyterHub, we are all learning more about its internal API and general system design.
This post contains only two class diagrams created with PyCharm. One is actually a SQLAlchemy ORM diagram, below.
This is the part 3, in a series of posts about Cylc
internals. The part 1
had the beginning of the workflow. part 2
documented from the moment the method
configure() is called. This post will
continue right after the
continue() method returns, going on with the next method: