I’ve been using JSONLint for every JSON prettifier needs. This site is super because it even format invalid JSON. For example, standard JSON prettifier won’t prettify JSON with key or value enclosed in single quote but this site does but having your local editor do quick prettifier if you have valid JSON or willing to quickly fix with find and replace.

Sublime is my go to text editor - taking note, prettify JSON etc. So let’s do that real quick.

Install Pretty JSON package. Assuming you are using Linux. If you are using other OS, key combination is different.

CTL + SHIFT + P > type “Install package” > Select Install package > Search for Pretty JSON > select it (this will install the package)

Okay now the package is installed, copy and paste into this UGLY JSON

[{"fav_fruit": "banana","fav_color":"red?","age":29}]

then hit CTL + ALT + J it will re-format to

[
  {
    "fav_fruit": "banana",
    "fav_color": "red?",
    "age": 29
  }
]

Whoohoo!

Cheers!

I tested out Stanza. English tokenizer and definately works. I ran quick test with Japanese lang and output was somewhat unexpected.

import stanza

# japanese "ja", for english model "en"

stanza.download("ja")
nlp = stanza.Pipeline("ja")
doc = nlp("皆さんおはようございます! ご機嫌いかがですか?")

for i, sentence in enumerate(doc.sentences):
    print(f"===== Sentence {i+1} tokens =====")
    print(*[f"word: {word.text}\t upos: {word.upos} xpos: {word.xpos}" for word in  sentence.words], sep="\n")

The output is:

===== Sentence 1 tokens =====
word: 皆さん	 upos: PRON xpos: NP
word: おは	 upos: VERB xpos: VV
word: よう	 upos: AUX xpos: AV
word: ござい	 upos: VERB xpos: VV
word: ます	 upos: AUX xpos: AV
word: !	 upos: PUNCT xpos: SYM
===== Sentence 2 tokens =====
word: ご	 upos: NOUN xpos: XP
word: 機	 upos: NOUN xpos: XS
word: 嫌い	 upos: NOUN xpos: NN
word: か	 upos: PART xpos: PF
word: が	 upos: ADP xpos: PS
word: です	 upos: AUX xpos: AV
word: か	 upos: PART xpos: PE
word: ?	 upos: PUNCT xpos: SYM

I’m not qualified to evaluate accuracy of POS etc but at least as far as tokenization goes, I would expect

["皆さん", "おはよう", "ござい", "ます", "!"]

and

["ご", "機嫌", "いかが", "です", "か". "?"]

Cheers!

See Useful keyboard shortcuts

Addition to the above:

  • Super + Space: Change input keyboard

  • Ctl + Q: Close an application window

  • Super + ↑ : Maxmize the application window.
  • Super → or ← : Move the application window to left right

```

Cheers!

Kafka is super popular and powerful data streaming platform but I’m surprised by lack of easy cli tool.

To find out version, there is no such thing as kafka --version you do:

$ find /opt/kafka/libs/ -name kafka_\* | head -1
/opt/kafka/libs/kafka_2.13-2.4.0-test-sources.jar

where 2.13 is scala version and 2.4.0 is kafka version.

One tool that I played with is lovely called kaf, “Modern CLI for Kafka”.

to list topics, you do:

sudo bin/kafka-topics.sh --list --bootstrap-server localhost:9092

whereas with kaf you do:

$ kaf topics  

to consume on console you do:

sudo bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test_topic

whereas with kaf you do:

kaf consume test_topic

just be able to omit –bootstrap-server etc makes you type less thus productive ;)

To install

go get github.com/birdayz/kaf/cmd/kaf

set auto-complete for zsh

echo 'source <(kaf completion zsh)' >> ~/.zshrc

for shell

echo 'source <(kaf completion bash)' >> ~/.bashrc

ADD and SELECT cluster ADD config for local

kaf config add-cluster local -b localhost:9092

This will create config file here:

~/.kaf/config 

Then you can SELECT to use this config

kaf config select-cluster 
Use the arrow keys to navigate: ↓ ↑ → ← 
? Select cluster: 
  ▸ local
    prod-kafka
    dev-kafka

LIST Kafka node

kaf node ls

LIST Kafka topics

kaf topics

Describe a topic

kaf topic describe ${topic}

List groups

kaf groups

Describe a group

You can see offset etc.

kaf group describe ${group}

LIST Kafka send message

echo '{"a": 123}' | kaf produce test_topic

LIST Kafka consume message

kaf consume test_topic --offset oldest|newest

Easy to install, easy to use, less typing and cluster def in yaml config. So why not use this!

Cheers!

Unused Docker image and container can quickly take up your disk spaces.

Let’s check disk usage

Summary of data used. You can add -v flag for detailed info.

$ docker system df
TYPE                TOTAL               ACTIVE              SIZE                RECLAIMABLE
Images              87                  1                   63.39GB             63.17GB (99%)
Containers          1                   0                   0B                  0B
Local Volumes       69                  1                   2.732GB             2.683GB (98%)
Build Cache         0                   0                   0B                  0B

Remove unused data

The following command will remove:

  • all stopped containers
  • all networks not used by at least one container
  • all dangling images (see below fo dangling images)
  • all build cache By default, volumes are not removed to prevent important data from being deleted.
$ docker system prune
WARNING! This will remove:
  - all stopped containers
  - all networks not used by at least one container
  - all dangling images
  - all dangling build cache

Are you sure you want to continue? [y/N] y
Deleted Containers:
...
Total reclaimed space: 13.91GB

Options:

  • --all, -a: Remove all unsued images not just dnaling ones.
  • --volumes: prune volumes.

What’s dangling images?

Docker images consist of multiple layers. Dangling images, are layers that have no relationship to any tagged images. They no longer serve a purpose and consume disk space.

You can list dangling iamges:

$ docker images -f dangling=true

Cheers!