Skip to content

Commit

Permalink
feat: docs information architecture (#496)
Browse files Browse the repository at this point in the history
## Description

Updating content and information architecture of the docs.

## Related Issue

Fixes #
<!-- or -->
Relates to #460 

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Other (security config, docs update, etc)

## Checklist before merging

- [x] Test, docs, adr added or updated as needed
- [x] [Contributor Guide
Steps](https://github.com/defenseunicorns/pepr/blob/main/CONTRIBUTING.md#submitting-a-pull-request)
followed

---------

Co-authored-by: Case Wylie <[email protected]>
  • Loading branch information
schaeferka and cmwylie19 authored Jan 22, 2024
1 parent 5c2f8c9 commit 58f79c3
Show file tree
Hide file tree
Showing 27 changed files with 600 additions and 85 deletions.
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Pepr is on a mission to save Kubernetes from the tyranny of YAML, intimidating g

- Zero-config K8s webhook mutations and validations
- Automatic leader-elected K8s resource watching
- Lightweight async key-value store backed by K8s for stateful operations with the [Pepr Store](./docs/store.md)
- Lightweight async key-value store backed by K8s for stateful operations with the [Pepr Store](./docs/030_user-guide/050_store.md)
- Human-readable fluent API for generating [Pepr Capabilities](#capability)
- A fluent API for creating/modifying/watching and server-side applying K8s resources via [Kubernetes Fluent Client](https://github.com/defenseunicorns/kubernetes-fluent-client)
- Generate new K8s resources based off of cluster resource changes
Expand All @@ -27,12 +27,12 @@ Pepr is on a mission to save Kubernetes from the tyranny of YAML, intimidating g
- Entire NPM ecosystem available for advanced operations
- Realtime K8s debugging system for testing/reacting to cluster changes
- Controller network isolation and tamper-resistent module execution
- Least-privilege [RBAC](https://github.com/defenseunicorns/pepr/blob/main/docs/rbac.md) generation
- Least-privilege [RBAC](./docs/030_user-guide/090_rbac.md) generation
- AMD64 and ARM64 support

## Example Pepr Action

This quick sample shows how to react to a ConfigMap being created or updated in the cluster. It adds a label and annotation to the ConfigMap and adds some data to the ConfigMap. It also creates a Validating Webhook to make sure the "pepr" label still exists. Finally, after the ConfigMap is created, it logs a message to the Pepr controller and creates or updates a separate ConfigMap with the [kubernetes-fluent-client](https://github.com/defenseunicorns/kubernetes-fluent-client) using server-side apply. For more details see [actions](./docs/actions.md) section.
This quick sample shows how to react to a ConfigMap being created or updated in the cluster. It adds a label and annotation to the ConfigMap and adds some data to the ConfigMap. It also creates a Validating Webhook to make sure the "pepr" label still exists. Finally, after the ConfigMap is created, it logs a message to the Pepr controller and creates or updates a separate ConfigMap with the [kubernetes-fluent-client](https://github.com/defenseunicorns/kubernetes-fluent-client) using server-side apply. For more details see [actions](./docs/030_user-guide/030_actions.md) section.

```ts
When(a.ConfigMap)
Expand Down Expand Up @@ -80,9 +80,9 @@ When(a.ConfigMap)
## Prerequisites

- [Node.js](https://nodejs.org/en/) v18.0.0+ (even-numbered releases only)
- To ensure compatability and optimal performance, it is recommended to use even-numbered releases of Node.js as they are stable releases and receive long-term support for three years. Odd-numbered releases are experimental and may not be supported by certain libraries utilized in Pepr.
- To ensure compatability and optimal performance, it is recommended to use even-numbered releases of Node.js as they are stable releases and receive long-term support for three years. Odd-numbered releases are experimental and may not be supported by certain libraries utilized in Pepr.

- [npm](https://www.npmjs.com/) v10.1.0+
- [npm](https://www.npmjs.com/) v10.1.0+

- Recommended (optional) tools:
- [Visual Studio Code](https://code.visualstudio.com/) for inline debugging and [Pepr Capabilities](#capability) creation.
Expand Down Expand Up @@ -113,13 +113,13 @@ https://user-images.githubusercontent.com/882485/230895880-c5623077-f811-4870-bb

A module is the top-level collection of capabilities. It is a single, complete TypeScript project that includes an entry point to load all the configuration and capabilities, along with their actions. During the Pepr build process, each module produces a unique Kubernetes MutatingWebhookConfiguration and ValidatingWebhookConfiguration, along with a secret containing the transpiled and compressed TypeScript code. The webhooks and secret are deployed into the Kubernetes cluster with their own isolated controller.

See [Module](./docs/module.md) for more details.
See [Module](./docs/030_user-guide/020_pepr-modules.md) for more details.

### Capability

A capability is set of related actions that work together to achieve a specific transformation or operation on Kubernetes resources. Capabilities are user-defined and can include one or more actions. They are defined within a Pepr module and can be used in both MutatingWebhookConfigurations and ValidatingWebhookConfigurations. A Capability can have a specific scope, such as mutating or validating, and can be reused in multiple Pepr modules.

See [Capabilities](./docs/capabilities.md) for more details.
See [Capabilities](./docs/030_user-guide/040_capabilities.md) for more details.

### Action

Expand All @@ -129,7 +129,7 @@ For example, an action could be responsible for adding a specific label to a Kub

There are both `Mutate()` and `Validate()` Actions that can be used to modify or validate Kubernetes resources within the admission controller lifecycle. There is also a `Watch()` Action that can be used to watch for changes to Kubernetes resources that already exist.

See [actions](./docs/actions.md) for more details.
See [actions](./docs/030_user-guide/030_actions.md) for more details.

## Logical Pepr Flow

Expand Down
35 changes: 35 additions & 0 deletions docs/010_overview/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Pepr Overview

## What is Pepr?

#### **_Type safe Kubernetes middleware for humans_**

<img align="right" width="40%" src="../_images/pepr.png" />

Pepr is on a mission to save Kubernetes from the tyranny of YAML, intimidating glue code, bash scripts, and other makeshift solutions. As a Kubernetes controller, Pepr empowers you to define Kubernetes transformations using TypeScript, without software development expertise thanks to plain-english configurations. Pepr transforms a patchwork of forks, scripts, overlays, and other chaos into a cohesive, well-structured, and maintainable system. With Pepr, you can seamlessly transition IT ops tribal knowledge into code, simplifying documentation, testing, validation, and coordination of changes for a more predictable outcome.

***Note: Pepr is still in active development so breaking changes may occur, but will be documented in release notes.***

## How Pepr Works

Pepr is a Kubernetes controller that watches for changes to the cluster and executes Pepr Modules in response to those changes. Pepr Modules are written in TypeScript and are compiled to WebAssembly (WASM) for execution in the Pepr Controller. Pepr Modules are written using the Pepr SDK which provides a fluent API for interacting with the Kubernetes API and other Pepr Modules. Pepr Modules are deployed to the Pepr Controller as Docker images and are executed in a sandboxed environment. Pepr Modules can be deployed to the Pepr Controller using the Pepr CLI or any other tool that can build and push Docker images.

## Features

### Out of the Box Features

- Zero-config K8s webhook mutations and validations
- Automatic leader-elected K8s resource watching
- Lightweight async key-value store backed by K8s for stateful operations with the Pepr Store
- Human-readable fluent API for generating Pepr Capabilities
- A fluent API for creating/modifying/watching and server-side applying K8s resources via Kubernetes Fluent Client
- Generate new K8s resources based off of cluster resource changes
- Perform other exec/API calls based off of cluster resources changes or any other arbitrary schedule
- Out of the box airgap support with Zarf
- Entire NPM ecosystem available for advanced operations
- Realtime K8s debugging system for testing/reacting to cluster changes
- Controller network isolation and tamper-resistent module execution
- Least-privilege RBAC generation
- AMD64 and ARM64 support

### Configurable Features
12 changes: 6 additions & 6 deletions docs/cli.md → docs/030_user-guide/010_pepr-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,7 @@ Update the current Pepr Module to the latest SDK version and update the global P

## `pepr dev`

Connect a local cluster to a local version of the Pepr Controller to do real-time debugging of your module. Note
the `pepr dev` assumes a K3d cluster is running by default. If you are working with Kind or another docker-based
K8s distro, you will need to pass the `--host host.docker.internal` option to `pepr dev`. If working with a remote
cluster you will have to give Pepr a host path to your machine that is reachable from the K8s cluster.
Connect a local cluster to a local version of the Pepr Controller to do real-time debugging of your module. Note the `pepr dev` assumes a K3d cluster is running by default. If you are working with Kind or another docker-based K8s distro, you will need to pass the `--host host.docker.internal` option to `pepr dev`. If working with a remote cluster you will have to give Pepr a host path to your machine that is reachable from the K8s cluster.

**Options:**

Expand All @@ -49,20 +46,23 @@ Deploy the current module into a Kubernetes cluster, useful for CI systems. Not

---

## pepr monitor
## `pepr monitor`

Monitor Validations for a given Pepr Module.

Usage:
Usage:

```bash
npx pepr monitor [options] <module-uuid>
```

**Options:**

- `-l, --log-level [level]` - Log level: debug, info, warn, error (default: "info")
- `-h, --help` - Display help for command

---

## `pepr build`

Create a [zarf.yaml](https://zarf.dev) and K8s manifest for the current module. This includes everything needed to deploy Pepr and the current module into production environments.
Expand Down
5 changes: 5 additions & 0 deletions docs/030_user-guide/020_pepr-modules.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Pepr Modules

## What is a Pepr Module?

A Pepr Module is a collection of files that can be used to create a new Pepr Project. A Pepr Module can be used to create a new Pepr Project by using the `pepr init` command. A Pepr Module can also be used to create a new Pepr Module by using the `pepr create-module` command.
7 changes: 2 additions & 5 deletions docs/actions.md → docs/030_user-guide/030_actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ For example, an action could be responsible for adding a specific label to a Kub

Actions are `Mutate()`, `Validate()`, or `Watch()`. Both Mutate and Validate actions run during the admission controller lifecycle, while Watch actions run in a separate controller that tracks changes to resources, including existing resources.

Let's look at some example actions that are included in the `HelloPepr` capability that is created for you when you [`pepr init`](./cli.md#pepr-init):
Let's look at some example actions that are included in the `HelloPepr` capability that is created for you when you [`pepr init`](./010_pepr-cli.md#pepr-init):

---

Expand Down Expand Up @@ -64,7 +64,4 @@ When(a.ConfigMap)
});
```

```
There are many more examples in the `HelloPepr` capability that you can use as a reference when creating your own actions. Note that each time you run [`pepr update`](./cli.md#pepr-update), Pepr will automatically update the `HelloPepr` capability with the latest examples and best practices for you to reference and test directly in your Pepr Module.
```

There are many more examples in the `HelloPepr` capability that you can use as a reference when creating your own actions. Note that each time you run [`pepr update`](./010_pepr-cli.md#pepr-update), Pepr will automatically update the `HelloPepr` capability with the latest examples and best practices for you to reference and test directly in your Pepr Module.
11 changes: 5 additions & 6 deletions docs/capabilities.md → docs/030_user-guide/040_capabilities.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,21 @@
# Capabilities
# Pepr Capabilities

A capability is set of related [actions](./actions.md) that work together to achieve a specific transformation or operation on Kubernetes resources. Capabilities are user-defined and can include one or more actions. They are defined within a Pepr module and can be used in both MutatingWebhookConfigurations and ValidatingWebhookConfigurations. A Capability can have a specific scope, such as mutating or validating, and can be reused in multiple Pepr modules.
A capability is set of related [actions](./030_actions.md) that work together to achieve a specific transformation or operation on Kubernetes resources. Capabilities are user-defined and can include one or more actions. They are defined within a Pepr module and can be used in both MutatingWebhookConfigurations and ValidatingWebhookConfigurations. A Capability can have a specific scope, such as mutating or validating, and can be reused in multiple Pepr modules.

When you [`pepr init`](./cli.md#pepr-init), a `capabilities` directory is created for you. This directory is where you will define your capabilities. You can create as many capabilities as you need, and each capability can contain one or more actions. Pepr also automatically creates a `HelloPepr` capability with a number of example actions to help you get started.
When you [`pepr init`](./010_pepr-cli.md#pepr-init), a `capabilities` directory is created for you. This directory is where you will define your capabilities. You can create as many capabilities as you need, and each capability can contain one or more actions. Pepr also automatically creates a `HelloPepr` capability with a number of example actions to help you get started.

## Creating a Capability

Defining a new capability can be done via a [VSCode Snippet](https://code.visualstudio.com/docs/editor/userdefinedsnippets) generated during [`pepr init`](./cli.md#pepr-init).
Defining a new capability can be done via a [VSCode Snippet](https://code.visualstudio.com/docs/editor/userdefinedsnippets) generated during [`pepr init`](./010_pepr-cli.md#pepr-init).

1. Create a new file in the `capabilities` directory with the name of your capability. For example, `capabilities/my-capability.ts`.

1. Open the new file in VSCode and type `create` in the file. A suggestion should prompt you to generate the content from there.

https://user-images.githubusercontent.com/882485/230897379-0bb57dff-9832-479f-8733-79e103703135.mp4
[https://user-images.githubusercontent.com/882485/230897379-0bb57dff-9832-479f-8733-79e103703135.mp4](https://user-images.githubusercontent.com/882485/230897379-0bb57dff-9832-479f-8733-79e103703135.mp4)

_If you prefer not to use VSCode, you can also modify or copy the `HelloPepr` capability to meet your needs instead._


## Reusable Capabilities

Pepr has an NPM org managed by Defense Unicorns, `@pepr`, where capabilities are published for reuse in other Pepr Modules. You can find a list of published capabilities [here](https://www.npmjs.com/search?q=@pepr).
Expand Down
7 changes: 4 additions & 3 deletions docs/store.md → docs/030_user-guide/050_store.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Pepr Store: A Lightweight Key-Value Store for Pepr Modules
# Pepr Store

The nature of admission controllers and general watch operations (the `Mutate`, `Validate` and `Watch` actions in Pepr) make some types of complex and long-running operations difficult. There are also times when you need to share data between different actions. While you could manually create your own K8s resources and manage their cleanup, this can be very hard to track and keep performant at scale.
## A Lightweight Key-Value Store for Pepr Modules

The nature of admission controllers and general watch operations (the `Mutate`, `Validate` and `Watch` actions in Pepr) make some types of complex and long-running operations difficult. There are also times when you need to share data between different actions. While you could manually create your own K8s resources and manage their cleanup, this can be very hard to track and keep performant at scale.

The Pepr Store solves this by exposing a simple, [Web Storage API](https://developer.mozilla.org/en-US/docs/Web/API/Storage)-compatible mechanism for use within capabilities. Additionally, as Pepr runs multiple replicas of the admission controller along with a watch controller, the Pepr Store provides a unique way to share data between these different instances automatically.

Expand All @@ -11,7 +13,6 @@ Each Pepr Capability has a `Store` instance that can be used to get, set and del
- **Asynchronous Key-Value Store**: Provides an asynchronous interface for storing small amounts of data, making it ideal for sharing information between various actions and capabilities.
- **Web Storage API Compatibility**: The store's API is aligned with the standard [Web Storage API](https://developer.mozilla.org/en-US/docs/Web/API/Storage), simplifying the learning curve.
- **Real-time Updates**: The `.subscribe()` and `onReady()` methods enable real-time updates, allowing you to react to changes in the data store instantaneously.

- **Automatic CRD Management**: Each Pepr Module has its data stored within a single Custom Resource Definition (CRD) that is automatically created upon deployment.
- **Efficient Operations**: Pepr Store uses Kubernetes watches, batch processing, and patch operations to make read and write operations as efficient as possible.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Importing Custom Resources
# Custom Resources

## Importing Custom Resources

The [Kubernetes Fluent Client](https://github.com/defenseunicorns/kubernetes-fluent-client) supports the creation of TypeScript typings directly from Kubernetes Custom Resource Definitions (CRDs). The files it generates can be directly incorporated into Pepr capabilities and provide a way to work with strongly-typed CRDs.

For example (below), Istio CRDs can be imported and used as though they were intrinsic Kubernetes resources.


## Generating TypeScript Types from CRDs

Using the kubernetes-fluent-client to produce a new type looks like this:
Expand Down Expand Up @@ -120,7 +120,7 @@ RegisterKind(ProxyConfig, {

## Using new types

The generated types can be imported into Pepr directly, _there is no additional logic needed to make them to work_.
The generated types can be imported into Pepr directly, _there is no additional logic needed to make them to work_.

```typescript
import { Capability, K8s, Log, a, kind } from "pepr";
Expand Down
13 changes: 6 additions & 7 deletions docs/OnSchedule.md → docs/030_user-guide/080_onschedule.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# OnSchedule

The `OnSchedule` feature allows you to schedule and automate the execution of specific code at predefined intervals or schedules. This feature is designed to simplify recurring tasks and can serve as an alternative to traditional CronJobs. This code is designed to be run at the top level on a Capability, not within a function like `When`.
The `OnSchedule` feature allows you to schedule and automate the execution of specific code at predefined intervals or schedules. This feature is designed to simplify recurring tasks and can serve as an alternative to traditional CronJobs. This code is designed to be run at the top level on a Capability, not within a function like `When`.

> **Note -** To use this feature in dev mode you MUST set `PEPR_WATCH_MODE="true"`. This is because the scheduler only runs on the watch controller and the watch controller is not started by default in dev mode.
For example: `PEPR_WATCH_MODE="true" npx pepr dev`

## Best Practices

`OnSchedule` is designed for targeting intervals equal to or larger than 30 seconds due to the storage mechanism used to archive schedule info.

## Usage

Create a recurring task execution by calling the OnSchedule function with the following parameters:
Expand All @@ -18,15 +18,14 @@ Create a recurring task execution by calling the OnSchedule function with the fo

**every** - An integer that represents the frequency of the schedule in number of _units_.

**unit** - A string specifying the time unit for the schedule (e.g., `seconds`, `minute`, `minutes`, `hour`, `hours`).
**unit** - A string specifying the time unit for the schedule (e.g., `seconds`, `minute`, `minutes`, `hour`, `hours`).

**startTime** - (Optional) A UTC timestamp indicating when the schedule should start. All date times must be provided in GMT. If not specified the schedule will start when the schedule store reports ready.

**run** - A function that contains the code you want to execute on the defined schedule.
**run** - A function that contains the code you want to execute on the defined schedule.

**completions** - (Optional) An integer indicating the maximum number of times the schedule should run to completion. If not specified the schedule will run indefinitely.


## Examples

Update a ConfigMap every 30 seconds:
Expand Down Expand Up @@ -73,7 +72,7 @@ OnSchedule({
});
```

## Advantages
## Advantages

- Simplifies scheduling recurring tasks without the need for complex CronJob configurations.
- Provides flexibility to define schedules in a human-readable format.
Expand Down
Loading

0 comments on commit 58f79c3

Please sign in to comment.