Website • Doc

Read this in other languages.

DataKit is an open source, integrated data collection agent, which provides full platform (Linux/Windows/macOS) support and has comprehensive data collection capability, covering various scenarios such as host, container, middleware, tracing, logging and security inspection.

Key Features

• Support collection of metrics, logging and tracing
• Fully support Kubernetes ecology
• Pipeline: Simple structured data extraction
• Supports third-party data import:
  • Telegraf
  • Prometheus
  • Statsd
  • Fluentd
  • Filebeats
  • Function
  • Tracing
    • OpenTelemetry
    • DDTrace
    • Zipkin
    • Jaeger
    • Skywalking

Changelog

All DataKit changelog refers to here.

Minimal Requirements

Install DataKit

We can directly obtain the DataKit installation command from guance cloud. Most of the installation commands seems like that:

• Linux & Mac

DK_DATAWAY="https://openway.guance.com?token=<YOUR-TOKEN>" bash -c "$(curl -L https://static.guance.com/datakit/install.sh)"

• Windows

Remove-Item -ErrorAction SilentlyContinue Env:DK_*;
$env:DK_DATAWAY="https://openway.guance.com?token=<YOUR-TOKEN>";
Set-ExecutionPolicy Bypass -scope Process -Force;
Import-Module bitstransfer;
start-bitstransfer -source https://static.guance.com/datakit/install.ps1 -destination .install.ps1;
powershell .install.ps1;
Remove-Item .install.ps1;

• Kubernetes DaemonSet

For more documentations about DataKit installation, see here.

Build From Source

DataKit building relies on some external tools/libs, we must install them all before compile the source code.

• We recommend to build source on Ubuntu 20.04+, other linux distribition may failed to install these dependencies. We do not support build DataKit on Windows.
• Please build the project with make, we haven't testing with Golang/VSCode IDEs

Setup Golang

Install and setup Golang(1.18.3+):

export GOPRIVATE=gitlab.jiagouyun.com/*
export GOPROXY=https://goproxy.cn,direct
export GOPATH=~/go            # depends on your local settings
export GOROOT=~/golang-1.18.3 # depends on your local settings
export PATH=$GOROOT/bin:~/go/bin:$PATH

Install other tools

!!! Do not install these dependencies under datakit source code dir.

• make: apt-get install make
• gcc: apt-get install gcc
• gcc-multilib: apt-get install -y gcc-multilib
• tree: apt-get install tree
• packr2: go install github.com/gobuffalo/packr/v2/packr2@v2.8.3
• goyacc: go install golang.org/x/tools/cmd/goyacc@latest
• lint related:
  • lint: go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.46.2
• eBPF related:
  • clang 10.0+: apt-get install clang
  • llvm 10.0+: apt-get install llvm
  • kernel headers
    • apt: apt-get install -y linux-headers-$(uname -r)
• Documentation exporting:
  • waque 1.13.1+

Build

1. Clone code

$ mkdir -p $GOPATH/src/gitlab.jiagouyun.com/cloudcare-tools
$ cd $GOPATH/src/gitlab.jiagouyun.com/cloudcare-tools

$ git clone https://github.com/GuanceCloud/datakit.git   # may be blocked by GFW
$ git clone https://jihulab.com/guance-cloud/datakit.git # jihulab mirror

$ cd datakit

2. Building

make

If building ok, all binaries are generated under dist:

dist
├── [4.0K]  datakit-linux-amd64
│   ├── [ 72M]  datakit
│   └── [4.0K]  externals
│       ├── [ 14M]  logfwd
│       └── [10.0M]  oracle
├── [4.0K]  local
│   ├── [ 26M]  installer-linux-amd64
│   └── [ 228]  version
└── [4.0K]  standalone
    └── [4.0K]  datakit-ebpf-linux-amd64
		        └── [ 38M]  datakit-ebpf

We can build all platforms(Linux/Mac/Windows) with following command:

make testing

Basic Usage

We can use help command to see more usage of DataKit:

datakit help

# Or

./dist/datakit-linux-amd64/datakit help

Contributing

Before contributing, check out some guideline of DataKit:

• Read architecure introduciton
• Read development guideline

Full Documentation

For full documents of DataKit, see

• DataKit Doc

License