Skip to content

feat(new source): Initial kubernetes_logs implementation#2653

Merged
MOZGIII merged 118 commits into
masterfrom
k8s-impl
Jul 22, 2020
Merged

feat(new source): Initial kubernetes_logs implementation#2653
MOZGIII merged 118 commits into
masterfrom
k8s-impl

Conversation

@MOZGIII

@MOZGIII MOZGIII commented May 20, 2020

Copy link
Copy Markdown
Contributor

This PR implements Kubernetes source according to #2222.
There are a few things are different from the RFC:

  • the name for the source is kubernetes_logs - it's wasn't properly addressed at the RFC, but the logs and other source kinds for the kubernetes would very likely have very different internal design, so we should probably keep them as separate units; to disambiguate from the very beginning I modified the name to be kubernetes_logs;
  • had to bump MSKV to 1.15 - watch bookmarks are introduced in that version, and this is what we want to have to avoid extra desync issues.

Done:

To do (before the merge):

To do (after the merge):

External refs:


The overall approach was to build highly composable components, so that we can reuse them for further work - like for adding a sidecar-oriented transform for pod annotation, or exclusions based on namespace labels.

@github-actions

github-actions Bot commented May 20, 2020

Copy link
Copy Markdown
Contributor

Great PR! Please pay attention to the following items before merging:

Files matching Cargo.lock:

  • Has at least one other team member approved the dependency changes?

This is an automatically generated QA checklist based on modified files

@MOZGIII MOZGIII force-pushed the k8s-impl branch 15 times, most recently from ac2abb3 to 6fe6fdf Compare May 22, 2020 05:52
@MOZGIII

MOZGIII commented May 25, 2020

Copy link
Copy Markdown
Contributor Author

The PR is ready for review, please go ahead with it!
There are still a few things left, I'm going to work on those this week. This shouldn't interfere with the review.

@MOZGIII MOZGIII marked this pull request as ready for review May 25, 2020 09:00
@MOZGIII MOZGIII force-pushed the k8s-impl branch 4 times, most recently from 1c78df3 to 4997b3e Compare May 25, 2020 12:05
@Hoverbear

Copy link
Copy Markdown
Contributor

I am so pumped!

Comment thread skaffold/docker/Dockerfile Outdated
Comment thread src/event/mod.rs
Comment thread src/internal_events/kubernetes_logs.rs
Comment thread src/kubernetes/client/config/in_cluster.rs Outdated
MOZGIII added 28 commits July 22, 2020 00:56
Signed-off-by: MOZGIII <mike-n@narod.ru>
…d.rs

Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Apparently it breaks everything. E2E tests start failing.
We need a better way.

This reverts commit a4c2a7d.

Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
It was controversial, and so this test is added.

Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
This should've been fixed by now, but it's still causing issues with older
k8s versions that we want to support.

Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
This reverts commit c0e1695.

Signed-off-by: MOZGIII <mike-n@narod.ru>
No actual changes there compared to upstream, just for preservation

Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

6 participants