Kafka is super popular and powerful data streaming platform but I’m surprised by lack of easy cli tool.

To find out version, there is no such thing as kafka --version you do:

$ find /opt/kafka/libs/ -name kafka_\* | head -1
/opt/kafka/libs/kafka_2.13-2.4.0-test-sources.jar

where 2.13 is scala version and 2.4.0 is kafka version.

One tool that I played with is lovely called kaf, “Modern CLI for Kafka”.

to list topics, you do:

sudo bin/kafka-topics.sh --list --bootstrap-server localhost:9092

whereas with kaf you do:

$ kaf topics  

to consume on console you do:

sudo bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test_topic

whereas with kaf you do:

kaf consume test_topic

just be able to omit –bootstrap-server etc makes you type less thus productive ;)

To install

go get github.com/birdayz/kaf/cmd/kaf

set auto-complete for zsh

echo 'source <(kaf completion zsh)' >> ~/.zshrc

for shell

echo 'source <(kaf completion bash)' >> ~/.bashrc

ADD and SELECT cluster ADD config for local

kaf config add-cluster local -b localhost:9092

This will create config file here:

~/.kaf/config 

Then you can SELECT to use this config

kaf config select-cluster 
Use the arrow keys to navigate: ↓ ↑ → ← 
? Select cluster: 
  ▸ local
    prod-kafka
    dev-kafka

LIST Kafka node

kaf node ls

LIST Kafka topics

kaf topics

Describe a topic

kaf topic describe ${topic}

List groups

kaf groups

Describe a group

You can see offset etc.

kaf group describe ${group}

LIST Kafka send message

echo '{"a": 123}' | kaf produce test_topic

LIST Kafka consume message

kaf consume test_topic --offset oldest|newest

Easy to install, easy to use, less typing and cluster def in yaml config. So why not use this!

Cheers!

Unused Docker image and container can quickly take up your disk spaces.

Let’s check disk usage

Summary of data used. You can add -v flag for detailed info.

$ docker system df
TYPE                TOTAL               ACTIVE              SIZE                RECLAIMABLE
Images              87                  1                   63.39GB             63.17GB (99%)
Containers          1                   0                   0B                  0B
Local Volumes       69                  1                   2.732GB             2.683GB (98%)
Build Cache         0                   0                   0B                  0B

Remove unused data

The following command will remove:

• all stopped containers
• all networks not used by at least one container
• all dangling images (see below fo dangling images)
• all build cache By default, volumes are not removed to prevent important data from being deleted.

$ docker system prune
WARNING! This will remove:
  - all stopped containers
  - all networks not used by at least one container
  - all dangling images
  - all dangling build cache

Are you sure you want to continue? [y/N] y
Deleted Containers:
...
Total reclaimed space: 13.91GB

Options:

• --all, -a: Remove all unsued images not just dnaling ones.
• --volumes: prune volumes.

What’s dangling images?

Docker images consist of multiple layers. Dangling images, are layers that have no relationship to any tagged images. They no longer serve a purpose and consume disk space.

You can list dangling iamges:

$ docker images -f dangling=true

Cheers!

Create tar.gz archives

tar czf new-tar-file-name.tar.gz file-or-folder-to-archive

c - create new archive z - compress the archive using gzip f - use archive file

Extract archives

tar -xzf tar-file-name.tar.gz

x - extract the archive. z - uncompress the archive using gzip. f - use archive file.

Let’s do it.

I only want to compress 3 files. When I extract archive what I want to see is files in the directory.

$ tree file_dir/
file_dir/
├── file1.txt
├── file2.txt
└── file3.txt

$ cd file_dir
$ tar czf files.tar.gz .
tar: .: file changed as we read it

# moved files.tar.gz to different directory
$ tar -xzf files.tar.gz 
$ ls
file1.txt  file2.txt  file3.txt  file_dir  files.tar.gz

Cheers!

What is GraalVM?

GraalVM is a high-performance, embeddable, polyglot virtual machine for running applications written in JavaScript, Python, Ruby, R, JVM-based languages like Java, Scala, Kotlin, and LLVM-based languages such as C and C++.

Here is the Official doc link

Hmm… Okay, I have to see it.

Let’s install

Below is the way I installed GraalVM Community Edition on Ubuntu 18.04. For other platform, the official doc installation guide is here.

# update this number to latest version from here: https://github.com/oracle/graal/releases
version=1.0.0-rc15
wget https://github.com/oracle/graal/releases/download/vm-${version}/graalvm-ce-${version}-linux-amd64.tar.gz
tar -xvzf graalvm-ce-${version}-linux-amd64.tar.gz
# clean up
rm graalvm-ce-${version}-linux-amd64.tar.gz
# to wherever you want. 
mv graalvm-ce-1.0.0-rc15/ ~/bin/graalvm
# if you want to make it permanent, put this in your bashrc 
export PATH=$HOME/graalvm/bin:$PATH

Now that your graalvm/bin in your path, you’ll get the GraalVM versions of those runtimes.

before:

$ js --version
v8.10.0

now:

$ js --version
Graal JavaScript 1.0 (GraalVM CE Native 1.0.0-rc15)

Now let’s play with the Polyglot Shell

The Polyglot Shell is useful to quickly test interactive Graal languages. This is how you can start it:

$ polyglot --jvm --shell
GraalVM MultiLanguage Shell 1.0.0-rc15
Copyright (c) 2013-2019, Oracle and/or its affiliates
  JavaScript version 1.0
Usage: 
  Use Ctrl+L to switch language and Ctrl+D to exit.
  Enter -usage to get a list of available commands.
js> 

Ah, it is listing only JavaScript. Got to install “python component” to run Python

gu install python

Now invoke it again

$ polyglot --jvm --shell
GraalVM MultiLanguage Shell 1.0.0-rc15
Copyright (c) 2013-2019, Oracle and/or its affiliates
  JavaScript version 1.0
  Python version 3.7.0
Usage: 
  Use Ctrl+L to switch language and Ctrl+D to exit.
  Enter -usage to get a list of available commands.
js> console.log('hello')
hello
python> print('hello')
hello
python> 

Oh yay!

For just python interactive shell

$ graalpython
Python 3.7.0 (Thu Apr 04 19:04:32 UTC 2019)
[GraalVM CE, Java 1.8.0_202] on linux
Type "help", "copyright", "credits" or "license" for more information.
Please note: This Python implementation is in the very early stages, and can run little more than basic benchmarks at this point.
>>> print('hello')
hello
>>> 

Debugger?

Debugger is available to all the GraalVM languages.

Let’s play with it. Prep the FizzBuzz program in python fizzbuzz.py

def fizzbuzz(n):
    if n % 3 == 0 and n % 5 ==0:
        return 'FizzBuzz'
    if n % 3 == 0:
        return 'Fizz'
    if n % 5 == 0:
        return 'Buzz'
    return n


if __name__ == '__main__':
    for n in range(1, 21):
        print(fizzbuzz(n))

Let’s start debugger

$ graalpython --jvm --inspect fizzbuzz.py
Debugger listening on port 9229.
To start debugging, open the following URL in Chrome:
    chrome-devtools://devtools/bundled/js_app.html?ws=127.0.0.1:9229/2c9f9fb0-8067ef52f64c8

1. Open the URL in Chrome
2. Click “Add folder to workspace” on the left to navigate the folder your fizzbuzz.py is located
3. Open fizzbuzz.py from Workspace and place breakpoint then resume script execution

Wow, that’s kind of impressive!

Web app in multiple languages

So this example is from this article Let’s create polyglot.js

const express = require('express')
const app = express()

const BigInteger = Java.type('java.math.BigInteger')

app.get('/', function (req, res) {
  var text = 'Hello World from Graal.js!<br> '

  // Using Java standard library classes
  text += BigInteger.valueOf(10).pow(100)
          .add(BigInteger.valueOf(43)).toString() + '<br>'

  // Using R interoperability to create graphs
  text += Polyglot.eval('R',
    `svg();
     require(lattice);
     x <- 1:100
     y <- sin(x/10)
     z <- cos(x^1.3/(runif(1)*5+10))
     print(cloud(x~y*z, main="cloud plot"))
     grDevices:::svg.off()
    `);

  res.send(text)
})

app.listen(3000, function () {
  console.log('Example app listening on port 3000!')
})

Let’s install dependencies:

First r component

gu install r

Next npm package express

npm install express

Let’s start the app

node --polyglot --jvm polyglot.js

Ta-da!

Cheers!

How to determine my public IP address assigned by the ISP via CLI

Use dig command

dig +short myip.opendns.com @resolver1.opendns.com

Use 3rd party

curl ifconfig.me

Cheers!