Menu

Home

Spring Cloud encrypted values and Spring PropertySources

kinow @ Apr 14, 2017 11:21:03

As I could not find any documentation for that, I decided to write it as a note to myself in case I use the encryption and decryption with Spring Cloud again.

In Spring and Spring Boot, you normally have multiple sources of properties, like multiple properties files, environment properties and variables, and so it goes. In the Spring API, these are represented as PropertySource‘s.

In a Spring Boot application, you would be used to overriding certain properties by defining environments and using an application-production.properties file, or overriding values with environment properties.

This is common in Spring Boot applications deployed to Amazon Elastic Beanstalk.

Some time ago another team at work found that overriding did not always work when you have encrypted values in your properties files. Even if you specified new values in the Amazon Elastic Beanstalk application configuration.

Yesterday, while debugging the issue and reading Spring Cloud source code, I found its EnvironmentDecryptApplicationInitializer.

It basically iterates through all loaded property sources, looking for values that start with {cipher}. Then it calls the Spring Security TextEncryptor defined in the application.

Finally, it creates a new property source, called decrypted, with the decrypted values. So when your application looks for a property called XPTO, and if it has been encrypted, it will find the value in the decrypted propery source, regardless of whether you tried to override it or not.

# Property sources listed in Eclipse IDE

[
  servletConfigInitParams,
  servletContextInitParams,
  systemProperties,
  systemEnvironment,
  random,
  applicationConfigurationProperties,
  springCloudClientHostInfo,
  defaultProperties
]

# When using encrypted values

[
  decrypted, <-------- created by Spring Cloud, with decrypted values. Prepended to the list of property sources
  servletConfigInitParams,
  servletContextInitParams,
  systemProperties,
  systemEnvironment,
  random,
  applicationConfigurationProperties,
  springCloudClientHostInfo,
  defaultProperties
]

So in case you have encrypted values in your Spring application (and you are using Spring Cloud, of course) remember that these values will have higher priority, and can only be overriden by other encrypted values.

♥ Open Source

Fixing Qt warning "QLayout: Attempting to add QLayout "" to QWidget "", which already has a layout"

kinow @ Apr 02, 2017 12:01:03

If you ever started Krita 3.x in your command line, and had a look at the console output, you may noticed the following warning.

QLayout: Attempting to add QLayout “” to QWidget “”, which already has a layout

Krita recently announced the release of 3.1.3-alpha-2, and while testing I saw this warning and decided to investigate why this warning happens.

There was already a similar question posted on StackOverflow. And the best answer’s initial paragraph gave me a hint of what to look for.

When you assign a widget as the parent of a QLayout by passing it into the constructor, the layout is automatically set as the layout for that widget. In your code you are not only doing this, but explicitly calling setlayout(). This is no problem when when the widget passed is the same. If they are different you will get an error because Qt tries to assign the second layout to your widget which has already had a layout set.

So, somewhere in Krita code, there was a a QWidget being created, and layouts were being added to it more than once. To find where the issue was happening was quite easy. A breakpoint at main.cc where the application is initialized, then step through a few times, until the message appeared in the console.

Further investigation led me to the History docker (the one that shows undo steps) constructor.

    QVBoxLayout *vl = new QVBoxLayout(page); // layout being set to page
    m_undoView = new KisUndoView(this);
    vl->addWidget(m_undoView);
    QHBoxLayout *hl = new QHBoxLayout(page); // layout being set to page again
    hl->addSpacerItem(new QSpacerItem(10, 1,  QSizePolicy::Expanding, QSizePolicy::Fixed));
    m_bnConfigure = new QToolButton(page);
    m_bnConfigure->setIcon(KisIconUtils::loadIcon("configure"));
    connect(m_bnConfigure, SIGNAL(clicked(bool)), SLOT(configure()));
    hl->addWidget(m_bnConfigure);
    vl->addItem(hl);

    setWidget(page);
    setWindowTitle(i18n("Undo History"));

Here the QWidget created receives both QHBoxLayout and QVBoxLayout. Again, searching the Internet a little bit, then came across this post with a good example of a QWidget with QHBoxLayout and QVBoxLayout. Here’s what the constructor looks after the patch has been applied.

    QVBoxLayout *vl = new QVBoxLayout(page); // layout being set to page
    m_undoView = new KisUndoView(this);
    vl->addWidget(m_undoView);
    QHBoxLayout *hl = new QHBoxLayout();
    hl->addSpacerItem(new QSpacerItem(10, 1,  QSizePolicy::Expanding, QSizePolicy::Fixed));
    m_bnConfigure = new QToolButton(page);
    m_bnConfigure->setIcon(KisIconUtils::loadIcon("configure"));
    connect(m_bnConfigure, SIGNAL(clicked(bool)), SLOT(configure()));
    hl->addWidget(m_bnConfigure);
    vl->addItem(hl);
    vl->addLayout(hl); // horizontal layout added to the vertical layout

    setWidget(page);
    setWindowTitle(i18n("Undo History"));

That’s it. Learned something new in Qt. Not as important and useful as learning about signals and slots, but now I can focus on other warnings in the console output of Krita.

And you? Have you tested Krita 3.1.3 alpha already? What are you waiting for? :-)

♥ Open Source

Simulating less memory with ulimit

kinow @ Mar 26, 2017 11:14:03

These days I was trying to reproduce a bug in Krita where it would crash when a user copied group layers between windows. It appeared that the user was getting a segmentation fault due to the user’s computer running out of memory.

I could not reproduce the issue, but my computer has 16 GB. The first thing that came to my mind was to create a virtual machine with less memory to reproduce the issue. But I decided to spend some time looking for a simpler way of doing it.

Searching the web I found some suggestions that ulimit could work. After playing for a while with ulimit and htop, and verifying the amount of memory necessary to open two files in two windows in Krita, I came up with the following settings.

# 2550 mb in kb
ulimit -v 2550000

# 2 gb in kb
ulimit -m 2000000

# Confirm limits
ulimit -a

gdb $HOME/Development/cpp/workspace/krita_install/bin/krita

Then after copying a few layers from one window to another, I successfully reproduced the issue, and could include a backtrace in the Krita issue tracking system.

♥ Open Source

Using Google Natural Language API in an AWS Elastic Beanstalk application

kinow @ Mar 15, 2017 13:18:03

Besides providing an API for users and developers, Google also provides a series of (very well-written) client implementation, in several programming languages. That’s the case for Google Storage, Google Vision, and for Google Natural Language API’s.

I recently had to use the latter in a POC at work, and ran into an interesting issue with our AWS Elastic Beanstalk environment.

Google Language API requires that clients authenticate before sending requests. If you use their google-cloud-java, you can define the GOOGLE_APPLICATION_CREDENTIALS. This environment variable must point to a JSON file with the Google Language API credentials.

The issue is that while Google Language API (or more precisely google-auth-library-java) looks for an environment variable, in Elastic Beanstalk you are able to specify only system properties (unless you want to try something with ebextensions, maybe some JNI…).

A workaround for this issue in Google Natural Language API, is to create and pass a LanguageServiceSettings to your LanguageServiceClient. This settings object, when created, must be given a channel provider with a FixedCredentialsProvider.

Of course reading code is way easier than reading this workaround description.

// File: GoogleNaturalLanguageService.java
// ...
    // envvar or property used to specify the Google Application Credentials
    private final static String GOOGLE_APPLICATION_CREDENTIALS = "GOOGLE_APPLICATION_CREDENTIALS";

    /**
     * Google Natural Language API.
     */
    private LanguageServiceClient languageServiceClient;

    @PostConstruct
    public void init() throws Exception {
        // Elastic Beanstalk supports Properties, not Environment Variables.
        // Google credentials library will load
        // the JSON location for the service to authenticate from an envVar. So
        // we need to fix that here.
        String googleApplicationCredentials = System.getenv(GOOGLE_APPLICATION_CREDENTIALS);
        LOGGER.info(String.format("GOOGLE_APPLICATION_CREDENTIALS in environment variable: %s", googleApplicationCredentials));
        if (StringUtils.isBlank(googleApplicationCredentials)) {
            googleApplicationCredentials = System.getProperty(GOOGLE_APPLICATION_CREDENTIALS);
            LOGGER.info(String.format("GOOGLE_APPLICATION_CREDENTIALS in JVM property: %s", googleApplicationCredentials));
        }

        if (googleApplicationCredentials == null) {
            throw new RuntimeException("Could not locate GOOGLE_APPLICATION_CREDENTIALS variable!");
        }

        final LanguageServiceSettings languageServiceSettings;
        try (InputStream is = new FileInputStream(new File(googleApplicationCredentials))) {
            final GoogleCredentials myCredentials = GoogleCredentials
                    .fromStream(is)
                    .createScoped(
                            Collections.singleton("https://www.googleapis.com/auth/cloud-platform")
                    );
            final CredentialsProvider credentialsProvider = FixedCredentialsProvider.create(myCredentials);

            final InstantiatingChannelProvider channelProvider = LanguageServiceSettings
                    .defaultChannelProviderBuilder()
                    .setCredentialsProvider(credentialsProvider)
                    .build();
            languageServiceSettings = LanguageServiceSettings
                    .defaultBuilder()
                    .setChannelProvider(channelProvider)
                    .build();
        } catch (IOException ioe) {
            LOGGER.error(String.format("IO error creating Google NLP settings: %s", ioe.getMessage()), ioe);
            throw ioe;
        }

        // Create Google API client
        this.languageServiceClient = LanguageServiceClient.create(languageServiceSettings);
    }

    @PreDestroy
    public void destroy() throws Exception {
        if (LOGGER.isDebugEnabled()) {
            LOGGER.debug("Destroying Google NLP API client");
        }
        // Close Google API executors and channels
        this.languageServiceClient.close();
    }
// ...

That way you should be able to use the API with AWS Elastic Beanstalk without having to hack your environment to provide the GOOGLE_APPLICATION_CREDENTIALS environment variable.

An alternative would be google-auth-library-java to look for an environment variable and a system property. Or maybe Amazon AWS add a way to provide environment variables.

Note also that I included the @PostConstruct and @PreDestroy annotated methods. The API will start an executor thread pool, so if you do not want to risk to have problems re-deploying your application, then remember to close your streams.

♥ Open Source

Order of containers in Docker Compose

kinow @ Mar 14, 2017 21:22:03

In Docker Compose you are able to control the startup order of the containers via the depends_on statement. This is documented in Controlling startup order in Compose.

If you have a simple setup, with Tomcat and Postgres, sometimes Postgres will start first, but Compose will initialize Tomcat before Postgres has fully booted. When that happens, you may receive 401, 404, or other application errors.

You can fix it by combining depends_on with a healthcheck. For example:

# File: docker-compose.yml
version: '2.1'
services:
  db:
    container_name: twpg
    build:
      context: .
      dockerfile: Dockerfile.postgres
    restart: always
    ports:
      - "5432:5432"
    healthcheck:
      test: "pg_isready -h localhost -p 5432 -q -U postgres"
      interval: 10s
      timeout: 5s
      retries: 5

  web:
    container_name: twtc
    build:
      context: .
      dockerfile: Dockerfile.tomcat
    restart: always
    depends_on:
      db:
        condition: service_healthy
    links:
      - db
    ports:
      - "80:8080"

In the example docker-compose.yml, there are two containers, db and web. web is running a Tomcat, and db is running Postgres. Web depends on db (see depends_on), and uses a condition service_healthy. Which indicates it depends that that container is healthy.

The healthcheck entry under the db container settings define how to check whether Postgres is running or not. In this case, we are using pg_isready, which is available in the vanilla Postgres 9 container.

It will try 5 times, with a 10 seconds interval, and will time out after 5 seconds. You may have to tune these parameters for your application.

This code snippet is from a pull request submitted to Foxoncz/docker-thingworx.

♥ Open Source