moznion's tech blog

moznion's technical memo

gowrtr - a library that supports golang code generation

(Japanese article is here: https://moznion.hatenadiary.com/entry/2019/01/14/111719)

I published a library that supports golang code generation, this named gowrtr (pronunciation: go writer). This library has a number of generators for golang code generation.

github.com

As Synopsis shows you:

package main

import (
    "fmt"

    "github.com/moznion/gowrtr/generator"
)

func main() {
    generator := generator.NewRoot(
        generator.NewComment(" THIS CODE WAS AUTO GENERATED"),
        generator.NewPackage("main"),
        generator.NewNewline(),
    ).AddStatements(
        generator.NewFunc(
            nil,
            generator.NewFuncSignature("main"),
        ).AddStatements(
            generator.NewRawStatement(`fmt.Println("hello, world!")`),
        ),
    ).
        EnableGofmt("-s").
        EnableGoimports()

    generated, err := generator.Generate(0)
    if err != nil {
        panic(err)
    }
    fmt.Println(generated)
}

the above code generates the following go code:

// THIS CODE WAS AUTO GENERATED
package main

import "fmt"

func main() {
        fmt.Println("hello, world!")
}

This library works as above.

This example is really simple so it looks it increases the amount of code should write at first glance, but if you want to generate large-scale code, this library should be useful to do that (please refer to the following topic about immutability).

For more detail, please refer to the GoDoc. The documentation also contains examples.

This library has two peculiarities:

  • It can apply code formatters (i.e. gofmt and/or goimports) to the generated code
  • Each method of the library act as immutable

The first one, it can obtain the advantage that "the format of the generated code is unified" and "can check the syntax of the generated code" by applying gofmt. Moreover, it can get the merit that "there is no necessary to think and write about which library should be imported while writing the generator for code generation" with using goimports.

The second means, each method of the generators doesn't change the internal state implicitly so a user can take a snapshot of the generator on demand. This is useful to reuse and derive the code generator instance.


By the way, there is square/javapoet that is a library supports Java code generation in the Java world. That is a great library. gowrtr is inspired by javapoet.

Perhaps this library lacks features and/or notation supports. If so please feel free to let me know.

Enjoy.

aws-lambda-perl5-layer is now available

I've released aws-lambda-perl5-layer: this is an AWS Lambda runtime layer for Perl5.

github.com

This project makes you to be able to execute Perl5 on AWS Lambda. This project uses AWS Lambda Runtime API that was announced re:Invent 2018. For detail of the API, please refer to the following documentation:

And I've published layers so you can start to use Perl5 Lambda with the provided one right now. Provided ARNs are following:

  • arn:aws:lambda:${REGION}:652718333417:layer:perl-5_26-layer:1
  • arn:aws:lambda:${REGION}:652718333417:layer:perl-5_28-layer:1

Supported regions are following:

  • ap-northeast-1
  • ap-northeast-2
  • ap-south-1
  • ap-southeast-1
  • ap-southeast-2
  • ca-central-1
  • eu-central-1
  • eu-west-1
  • eu-west-2
  • eu-west-3
  • sa-east-1
  • us-east-1
  • us-east-2
  • us-west-1
  • us-west-2

Please see the repository documentation and moznion/aws-lambda-perl5-layer-example for usage.

github.com

Enjoy!

Docuss - A library to test with describing document for controller layer

I released a library "Docuss" for Java (this library requires Java 8 or later).
It is available in maven central.

maven central

This library works as below:

  • This library provides an inspection method to test the HTTP response. This library sends an HTTP request to target URI and it hands over the response to the inspector.
  • When test is passed, this library outputs the contents of the HTTP request and response *1 as any format by arbitrary method.

In short, this library provides a function to test HTTP request and response with describing their contents.
It is similar to autodoc, however docuss is more minimal (or cheaper?) than that.

Fundamental information and usage are written in the README and javadoc, the points that worked out are below;

  • Format of output is controllable
  • Method to output is controllable
  • It doesn't depend on any concrete HTTP client implementation

Format of output is controllable

This library can specify the format of output of documents.
It is easy to do that; you can specify the output format by writing formatter generator which outputs the formatter to convert contents of request and response to arbitrary format. Formatter generator must be implemented DocussFormatterGenerator interface.

Default, this library provides a formatter generator that generates YAML formatter.

Method to output is controllable

This library can specify the destination of the output. The way is just like formats’ one; you can specify the method to output by implementing the interface DocussPresenter.

Default, this library provides two presenters. One is a presenter to output to STDIN, another is a presenter to output to file with appending.

It doesn’t depend on any concrete HTTP client implementation

This library doesn’t depend on any concrete HTTP client implementation so if you want to use specific HTTP client, you can implement DocussHttpClient interface. Basically, the implementation of this interface bears the responsibility to implement the core of functions.

Default, this library supports the simple HTTP client layer implementation which is using Apache httpclient.

Conclusion

I think the way that outputs document from test code is right and natural. I want library to satisfy such function so implemented this.
I usually get out of breath when finished writing the production code and test code so it is difficult for me to write the document.
Even if it is in such a situation, we should write documents firmly so I think such library to support to write documents is necessary.

Enjoy.

*1:in this library, these are called "document"

sprint - Fluent, small and fast string formatter for Java

I published "sprint" version 0.0.2.

And it is also available on maven central. http://search.maven.org/#artifactdetails%7Cnet.moznion%7Csprint%7C0.0.2%7Cjar

Sprint is a string formatter library for Java.
Simple usage is below.

final Sprint sprint = new Sprint();
System.out.Println(sprint.ff("Hello: {}!", "John")); // => Hello: John!

Pros of this formatter library are following.

Fluent template

Sprint takes a fluent (or possibly loose?) template like "{}:{}". {} is a placeholder. This formatter fills arguments into placeholders which are corresponded.
There is no need to mind about correspondence between placeholders of a template and type of arguments. Sprint fills argument with calling Object#toString().

Fast formatting

Sprint parses template to pick up placeholders, and it constructs data structure according to parsed result. This formatter builds a string with scanning this data structure.
Sprint stores this parsed data structure with linking to template in ConcurrentHashMap which is in instance of Sprint, so when formatting with the same template, it uses corresponded pre-parsed data structure.
It means this formatter parses template only at once, so it can reduce redundant template parsing processing.

Result of benchmarking is below (result of jdk1.8.0_92, benchmark code is here). f:id:moznion:20160605223844p:plain

Small

Source code quantity is small and simple :)

Conclusion

Sprint is a fluent, small and fast string formatter for Java.
This formatter is a beta quality, it has potential for growth of performance.
Enjoy.

resque_exporter is out

This is an exporter of Prometheus for resque's queue status *1.
It aggregates and provides number of remained jobs for each queues.

Aggregation mechanism is simple. This exporter accesses to redis to aggregate queue status.

  1. Collect name of queues via <namespace>:queues entry (by using SMEMBERS)
  2. Get number of remained jobs for each queue via <namespace>:queue:<queue_name> entry (by using LLEN)

Exporter outputs metrics through HTTP, like so:

# HELP resque_jobs_in_queue Number of remained jobs in queue
# TYPE resque_jobs_in_queue gauge
resque_jobs_in_queue{queue_name="image_converting"} 0
resque_jobs_in_queue{queue_name="log_compression"} 0

It is useful to monitor and visualise status of queues by using prometheus.
Enjoy.

*1:it also supports resque compatible job queue system; e.g. jesque