Software Engineering Blog

Hadoop ecosystem, Scala, Java, Go, Micoroservices

Writing Internal DSL in Scala

Definition of DSL A DSL is a computer language, specialized for a specific domain. A DSL is essentially the opposite of a general programming languages(GPL), I mean every language that can be used to develop something. For example, Java, C#, and Scala are all GPL languages. With a DSL, we want only to solve a specific domain problem, so we design the language specifically for the domain. A DSL can’t be used outside of the domain, because it is not designed for flexibility.

Dev tools: ngrok

Have you ever wanted to show your work which is running on your local machine to your colleagues or friends? If they are far away of you, how will you show them? Will you upload your source code to github to share it with your friends? But what if you don’t like to share your source code with others? Will you run it on public or private cloud or something like that?

5 ways to customize Spring MVC JSON/XML output

To adhere to guidelines or requirements, API designers may want to control how JSON/XML responses are formatted. Spring Web makes use of Jackson to perform JSON/XML serialization. Therefore, to customize our output format, we must configure the Jackson processor. Spring Web offers XML-based or Java-based approaches to handling configuration. In this article, we will look at the Java-based configuration. Enable XML output First of all, if you have created your project through https://start.

Function Memorization in Go

Memoization is an optimization technique used to increase performance by storing the results of expensive function calls and returning the cached result when the same input occurs again. In this post I show how function memoization can be implemented in Go, in a pure functional manner. Based on wikipedia definition, a function may be considered a pure function if both of the following statements about the function hold: 1. The function always evaluates the same result value given the same argument value(s).

setcap Linux Command

Today I learned something new that I want to share with you. I knew that listening on port below 1024 requires special privilege, and to accomplish that you must be sudoers. But running applications with sudo is not a perfect way because that way the application can do almost anything to your operating system resulting unexpected results (you surely don’t want the application delete all your files by rm -rf).

Enhance Hadoop MapReduce Speed for small jobs

Introduction There are some circumstances when input of Hadoop’s MapReduce is relatively small. Consequently the overhead of allocating and running tasks in new containers outweighs the gain to be had in running them in parallel, compared to running them sequentially on one node. Such a job is said to be uberized, or run as an uber task. Enable uber optimization To enable uberized job, simply set mapreduce.job.ubertask.enable to true. But that is not sufficient.

Kafka Streams + SSE = Realtime web app

Last week I decided to dirty my hands with Kafka Streams. I wanted to write simple application with Kafka Streams more interesting than World Count. I decided to write a program that calculates top 5 using Kafka Streams. Imagine you have written a game, and you want to display top 5 users. You also need that calculating and displaying top 5 to be real time. You need whenever a user has hit a record, it automatically displays user record.

Publishing Text Messages to the Kafka Without Writing Any Code

Perhaps the ordinary way to produce messages to the Kafka is through the standard Kafka clients. But if you want to just produce text messages to the Kafak, there are simpler ways.In this tutorial I ’ll show you 3 ways of sending text messages to the Kafka. kafka-console-producer One way is through kafka-console-producer that is bundled with Kafka distribution. For example to send a file to the Kafka, you can write:

Docker Swarm (Part 2)

Introduction In the previous post, we ’ve learned how to create a cluster of machines using Docker Swarm, and how to ask Swarm to run 2 instances of Nginx on those machines.In this post, we will learn how to run our custom applications on the cluster. We ’re going to implement two services using Go, one of them is fontend service and the other is backend service. The user can interact only with frontend service.

Say Hello to Docker Swarm

Say Hello to Docker Swarm (part 1) In this tuturial I want to show you, how you can create cluster of machines using Docker Swarm and how to run your services on docker swarm.I assume you have basic knowledge of Docker.On part 2, we will implement our services on Golang, and will run them on docker swarm, so stay tuned. Install Docker Machine In this tutorial I use docker-machine to simulate physical machines.