Released go-json-ice: a code generator of JSON marshaler for tinygo
In tinygo, if the go code imports encoding/json package, it fails to compile. AFAIK, this is a known issue *1, so it seemed there is no standard way to marshal any struct to JSON string. There are some related issues like the following:
The point is if we would like to marshal a struct to JSON string, "serialize it by hand, carefully" is an only way for that, but it's a little too inconvenient so I made a code generator of JSON marshaler for that purpose; the generated marshaler doesn't have the dependency belongs to encoding/json.
This library parses the struct (i.e. its json custom struct tag) that to marshal beforehand and generates marshaler code according to the parsed result. There have been already similar implementations, such as mailru/easyjson, but they depend on encoding/json internally so it hasn't matched with my purpose.
For example, if you'd like to marshal the following struct, just put go:generate with that,
//go:generate json-ice --type=AwesomeStruct type AwesomeStruct struct { Foo string `json:"foo"` Bar string `json:"bar,omitempty"` }
then, it generates marshaler code as MarshalAwesomeStructAsJSON(s *AwesomeStruct) ([]byte, error), next you can marshal a struct to JSON with that:
marshaled, err := MarshalAwesomeStructAsJSON(&AwesomeStruct{
Foo: "buz",
Bar: "",
})
if err != nil {
log.Fatal(err)
}
fmt.Printf("%s\n", marshaled) // => {"foo":"buz"}
Therefore, it no longer needed to use reflection for marshaling, so tinygo can marshal struct easily. And also it resolves fields of marshaling target struct in advance, thus the performance gets better.
Of course, it has side effects. If the target field type was interface{}, it cannot marshal that because it can't resolve the actual type of the field dynamically. The type must be resolvable as static.
And as you know, tinygo can transpile to wasm, and the modules that are imported in wasm on running depends on what original code depends on. The generated code from this library is interested to keep that dependency as minimal as possible because there is the possibility to execute wasm in a super-restricted sandbox *2. As a result, the code only relies on strconv package.
I'm happy if you try this library!
FYI, this library only supports JSON marshaling. If you're looking for a way to unmarshal JSON in tinygo, I guess buger/jsonparser would help you for the purpose:
It does not rely on encoding/json, reflection or interface{}, the only real package dependency is bytes.
By the way,
//go:generate json-ice --type=DeepStruct type DeepStruct struct { Deep []map[string]map[string]map[string]map[string]string `json:"deep"` }
this library can generate the marshaler code for a struct that has deep and recursive field type like the above.
given := &DeepStruct{
Deep: []map[string]map[string]map[string]map[string]string{
{
"foo": {
"bar": {
"buz": {
"qux": "foobar",
},
},
},
},
{
"foofoo": {
"barbar": {
"buzbuz": {
"quxqux": "foobarfoobar",
},
},
},
},
},
}
marshaled, err := MarshalDeepStructAsJSON(given)
if err != nil {
log.Fatal(err)
}
log.Printf("[debug] %s", marshaled) // => {"deep":[{"foo":{"bar":{"buz":{"qux":"foobar"}}}},{"foofoo":{"barbar":{"buzbuz":{"quxqux":"foobarfoobar"}}}}]}
The generated code for that is the following :)
import "github.com/moznion/go-json-ice/serializer" func MarshalDeepStructAsJSON(s *DeepStruct) ([]byte, error) { buff := make([]byte, 1, 54) buff[0] = '{' if s.Deep == nil { buff = append(buff, "\"deep\":null,"...) } else { buff = append(buff, "\"deep\":"...) buff = append(buff, '[') for _, v := range s.Deep { if v == nil { buff = append(buff, "null"...) } else { buff = append(buff, '{') for mapKey, mapValue := range v { buff = serializer.AppendSerializedString(buff, mapKey) buff = append(buff, ':') if mapValue == nil { buff = append(buff, "null"...) } else { buff = append(buff, '{') for mapKey, mapValue := range mapValue { buff = serializer.AppendSerializedString(buff, mapKey) buff = append(buff, ':') if mapValue == nil { buff = append(buff, "null"...) } else { buff = append(buff, '{') for mapKey, mapValue := range mapValue { buff = serializer.AppendSerializedString(buff, mapKey) buff = append(buff, ':') if mapValue == nil { buff = append(buff, "null"...) } else { buff = append(buff, '{') for mapKey, mapValue := range mapValue { buff = serializer.AppendSerializedString(buff, mapKey) buff = append(buff, ':') buff = serializer.AppendSerializedString(buff, mapValue) buff = append(buff, ',') } if buff[len(buff)-1] == ',' { buff[len(buff)-1] = '}' } else { buff = append(buff, '}') } } buff = append(buff, ',') } if buff[len(buff)-1] == ',' { buff[len(buff)-1] = '}' } else { buff = append(buff, '}') } } buff = append(buff, ',') } if buff[len(buff)-1] == ',' { buff[len(buff)-1] = '}' } else { buff = append(buff, '}') } } buff = append(buff, ',') } if buff[len(buff)-1] == ',' { buff[len(buff)-1] = '}' } else { buff = append(buff, '}') } } buff = append(buff, ',') } if buff[len(buff)-1] == ',' { buff[len(buff)-1] = ']' } else { buff = append(buff, ']') } buff = append(buff, ',') } if buff[len(buff)-1] == ',' { buff[len(buff)-1] = '}' } else { buff = append(buff, '}') } return buff, nil }
*1:because of https://tinygo.org/lang-support/stdlib/#encoding-json
*2:i.e. not a browser runtime
Released sbt-spotless: an sbt plugin for Spotless
I've released sbt-spotless, this is an sbt plugin for Spotless.
Spotless is currently supporting plugins for maven and gradle, but it seems there had been no sbt's one, so I made this. This plugin works on sbt 1.3.x and legacy sbt 0.13.x *1.
The usage is quite simple. This plugin has already been published the Maven Central, so you just only put a dependency definition into your plugins.sbt:
addSbtPlugin("net.moznion.sbt" % "sbt-spotless" % "0.1.3")
next, configure your build.sbt for code formatter(s):
import net.moznion.sbt.spotless.config._ lazy val root = (project in file(".")) .settings( name := "Example", spotlessJava := JavaConfig( googleJavaFormat = GoogleJavaFormatConfig("1.7") ), )
finally, you can run sbt spotlessCheck and sbt spotlessApply commands to check and apply the code formatter.
And – this feature is only available on sbt 1.3.x – you can use checkOnCompile and applyOnCompile options like the following:
lazy val root = (project in file(".")) .settings( name := "Example", spotless := SpotlessConfig( checkOnCompile = true, // applyOnCompile = true, ), )
then, it checks or applies the code formatter automatically on compiling, automatically.
If you'd like to know configuration in detail, please refer to the following documentation: Configurations · moznion/sbt-spotless Wiki · GitHub
This plugin supports Java, Scala, Kotlin, Groovy, C++, and SQL formatters so far, but Spotless core supports various formatters furthermore. Please feel free to ask me through an issue or pull-request if you feel this should support another formatter.
Enjoy!
*1:but on sbt 0.13.x, the features are limited
Published proxy-protocol-js
(Japanese article is here: https://moznion.hatenadiary.com/entry/2019/04/30/131509 )
I had published a library that is named proxy-protocol-js.
This library recognizes the PROXY protocol and has responsibilities for building and parsing the protocol for JavaScript and TypeScript (for details of PROXY protocol, please refer to the following the spec: https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt ).
This library's strengths are the following:
- Supports V1 (text protocol) and V2 (binary protocol)
- Supports multipurpose functions
- To build a protocol payload
- To parse a protocol payload
- To identify the protocol version according to a protocol payload
- Better performance than existing implementations
- 2 times faster!
- The benchmarking result is here: https://github.com/moznion/proxy-protocol-js/blob/master/README.md#performance
- Supports TypeScript (i.e. this library is written in TypeScript)
- No extra dependencies
Enjoy!
gowrtr - a library that supports golang code generation
(Japanese article is here: https://moznion.hatenadiary.com/entry/2019/01/14/111719)
I published a library that supports golang code generation, this named gowrtr (pronunciation: go writer). This library has a number of generators for golang code generation.
As Synopsis shows you:
package main import ( "fmt" "github.com/moznion/gowrtr/generator" ) func main() { generator := generator.NewRoot( generator.NewComment(" THIS CODE WAS AUTO GENERATED"), generator.NewPackage("main"), generator.NewNewline(), ).AddStatements( generator.NewFunc( nil, generator.NewFuncSignature("main"), ).AddStatements( generator.NewRawStatement(`fmt.Println("hello, world!")`), ), ). EnableGofmt("-s"). EnableGoimports() generated, err := generator.Generate(0) if err != nil { panic(err) } fmt.Println(generated) }
the above code generates the following go code:
// THIS CODE WAS AUTO GENERATED package main import "fmt" func main() { fmt.Println("hello, world!") }
This library works as above.
This example is really simple so it looks it increases the amount of code should write at first glance, but if you want to generate large-scale code, this library should be useful to do that (please refer to the following topic about immutability).
For more detail, please refer to the GoDoc. The documentation also contains examples.
This library has two peculiarities:
- It can apply code formatters (i.e.
gofmtand/orgoimports) to the generated code - Each method of the library act as immutable
The first one, it can obtain the advantage that "the format of the generated code is unified" and "can check the syntax of the generated code" by applying gofmt. Moreover, it can get the merit that "there is no necessary to think and write about which library should be imported while writing the generator for code generation" with using goimports.
The second means, each method of the generators doesn't change the internal state implicitly so a user can take a snapshot of the generator on demand. This is useful to reuse and derive the code generator instance.
By the way, there is square/javapoet that is a library supports Java code generation in the Java world. That is a great library. gowrtr is inspired by javapoet.
Perhaps this library lacks features and/or notation supports. If so please feel free to let me know.
Enjoy.
aws-lambda-perl5-layer is now available
I've released aws-lambda-perl5-layer: this is an AWS Lambda runtime layer for Perl5.
This project makes you to be able to execute Perl5 on AWS Lambda. This project uses AWS Lambda Runtime API that was announced re:Invent 2018. For detail of the API, please refer to the following documentation:
- AWS Lambda Now Supports Custom Runtimes, and Enables Sharing Common Code Between Functions
- Custom AWS Lambda Runtimes - AWS Lambda
- AWS Lambda Runtime Interface - AWS Lambda
And I've published layers so you can start to use Perl5 Lambda with the provided one right now. Provided ARNs are following:
arn:aws:lambda:${REGION}:652718333417:layer:perl-5_26-layer:1arn:aws:lambda:${REGION}:652718333417:layer:perl-5_28-layer:1
Supported regions are following:
- ap-northeast-1
- ap-northeast-2
- ap-south-1
- ap-southeast-1
- ap-southeast-2
- ca-central-1
- eu-central-1
- eu-west-1
- eu-west-2
- eu-west-3
- sa-east-1
- us-east-1
- us-east-2
- us-west-1
- us-west-2
Please see the repository documentation and moznion/aws-lambda-perl5-layer-example for usage.
Enjoy!
Docuss - A library to test with describing document for controller layer
I released a library "Docuss" for Java (this library requires Java 8 or later).
It is available in maven central.
This library works as below:
- This library provides an inspection method to test the HTTP response. This library sends an HTTP request to target URI and it hands over the response to the inspector.
- When test is passed, this library outputs the contents of the HTTP request and response *1 as any format by arbitrary method.
In short, this library provides a function to test HTTP request and response with describing their contents.
It is similar to autodoc, however docuss is more minimal (or cheaper?) than that.
Fundamental information and usage are written in the README and javadoc, the points that worked out are below;
- Format of output is controllable
- Method to output is controllable
- It doesn't depend on any concrete HTTP client implementation
Format of output is controllable
This library can specify the format of output of documents.
It is easy to do that; you can specify the output format by writing formatter generator which outputs the formatter to convert contents of request and response to arbitrary format. Formatter generator must be implemented DocussFormatterGenerator interface.
Default, this library provides a formatter generator that generates YAML formatter.
Method to output is controllable
This library can specify the destination of the output. The way is just like formats’ one; you can specify the method to output by implementing the interface DocussPresenter.
Default, this library provides two presenters. One is a presenter to output to STDIN, another is a presenter to output to file with appending.
It doesn’t depend on any concrete HTTP client implementation
This library doesn’t depend on any concrete HTTP client implementation so if you want to use specific HTTP client, you can implement DocussHttpClient interface. Basically, the implementation of this interface bears the responsibility to implement the core of functions.
Default, this library supports the simple HTTP client layer implementation which is using Apache httpclient.
Conclusion
I think the way that outputs document from test code is right and natural. I want library to satisfy such function so implemented this.
I usually get out of breath when finished writing the production code and test code so it is difficult for me to write the document.
Even if it is in such a situation, we should write documents firmly so I think such library to support to write documents is necessary.
Enjoy.
*1:in this library, these are called "document"
sprint - Fluent, small and fast string formatter for Java
I published "sprint" version 0.0.2.
And it is also available on maven central. http://search.maven.org/#artifactdetails%7Cnet.moznion%7Csprint%7C0.0.2%7Cjar
Sprint is a string formatter library for Java.
Simple usage is below.
final Sprint sprint = new Sprint(); System.out.Println(sprint.ff("Hello: {}!", "John")); // => Hello: John!
Pros of this formatter library are following.
Fluent template
Sprint takes a fluent (or possibly loose?) template like "{}:{}". {} is a placeholder. This formatter fills arguments into placeholders which are corresponded.
There is no need to mind about correspondence between placeholders of a template and type of arguments. Sprint fills argument with calling Object#toString().
Fast formatting
Sprint parses template to pick up placeholders, and it constructs data structure according to parsed result. This formatter builds a string with scanning this data structure.
Sprint stores this parsed data structure with linking to template in ConcurrentHashMap which is in instance of Sprint, so when formatting with the same template, it uses corresponded pre-parsed data structure.
It means this formatter parses template only at once, so it can reduce redundant template parsing processing.
Result of benchmarking is below (result of jdk1.8.0_92, benchmark code is here).
Small
Source code quantity is small and simple :)
Conclusion
Sprint is a fluent, small and fast string formatter for Java.
This formatter is a beta quality, it has potential for growth of performance.
Enjoy.
