Skip to content

Commit 65adfc5

Browse files
abravermabraverm
committed
flake.nix
Package Sourcebot with Nix, NixOS module for deployment, integration test and microvm.
1 parent 769c4e0 commit 65adfc5

File tree

9 files changed

+1132
-0
lines changed

9 files changed

+1132
-0
lines changed

‎docs/docs/deployment-guide.mdx‎

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
---
2+
title: "Deployment guide"
3+
---
4+
5+
import SupportedPlatforms from '/snippets/platform-support.mdx'
6+
7+
## Container deployment
8+
9+
The following guide will walk you through the steps to deploy Sourcebot on your own infrastructure. Sourcebot is distributed as a [single docker container](/docs/overview#architecture) that can be deployed to a k8s cluster, a VM, or any platform that supports docker.
10+
11+
12+
<Note>Hit an issue? Please let us know on [GitHub](https://github.com/sourcebot-dev/sourcebot/issues/new/choose) or by [emailing us](mailto:team@sourcebot.dev).</Note>
13+
14+
<Steps>
15+
<Step title="Requirements">
16+
- Docker -> use [Docker Desktop](https://www.docker.com/products/docker-desktop/) on Mac or Windows.
17+
</Step>
18+
<Step title="Create a config.json">
19+
Create a `config.json` file that tells Sourcebot which repositories to sync and index:
20+
21+
```bash wrap icon="terminal" Create example config
22+
touch config.json
23+
echo '{
24+
"$schema": "https://raw.githubusercontent.com/sourcebot-dev/sourcebot/main/schemas/v3/index.json",
25+
"connections": {
26+
// comments are supported
27+
"starter-connection": {
28+
"type": "github",
29+
"repos": [
30+
"sourcebot-dev/sourcebot"
31+
]
32+
}
33+
}
34+
}' > config.json
35+
```
36+
37+
This config creates a single GitHub connection named `starter-connection` that specifies [Sourcebot](https://github.com/sourcebot-dev/sourcebot) as a repo to sync. [Learn more about the config file](/docs/configuration/config-file).
38+
</Step>
39+
40+
<Step title="Launch your instance">
41+
<Warning>If you're deploying Sourcebot behind a domain, you must set the [AUTH_URL](/docs/configuration/environment-variables) environment variable.</Warning>
42+
43+
44+
In the same directory as `config.json`, run the following command to start your instance:
45+
46+
``` bash icon="terminal" Start the Sourcebot container
47+
docker run \
48+
-p 3000:3000 \
49+
--pull=always \
50+
--rm \
51+
-v $(pwd):/data \
52+
-e CONFIG_PATH=/data/config.json \
53+
--name sourcebot \
54+
ghcr.io/sourcebot-dev/sourcebot:latest
55+
```
56+
57+
<Accordion title="Details">
58+
**This command**:
59+
- pulls the latest version of the `sourcebot` docker image.
60+
- mounts the working directory to `/data` in the container to allow Sourcebot to persist data across restarts, and to access the `config.json`. In your local directory, you should see a `.sourcebot` folder created that contains all persistent data.
61+
- runs any pending database migrations.
62+
- starts up all services, including the webserver exposed on port 3000.
63+
- reads `config.json` and starts syncing.
64+
</Accordion>
65+
66+
</Step>
67+
68+
<Step title="Complete onboarding">
69+
Navigate to `http://localhost:3000` and complete the onboarding flow.
70+
</Step>
71+
72+
<Step title="Done">
73+
You're all set! If you'd like to setup [Ask Sourcebot](/docs/features/ask/overview), configure a language model [provider](/docs/configuration/language-model-providers).
74+
</Step>
75+
</Steps>
76+
77+
78+
## NixOS deployment
79+
80+
<Note>Hit an issue? Please let us know on [GitHub discussions](https://github.com/sourcebot-dev/sourcebot/discussions/categories/support) or by [emailing us](mailto:team@sourcebot.dev).</Note>
81+
82+
<Steps>
83+
<Step title="Flake.nix input">
84+
Add the Sourcebot flake as an input to your NixOS configuration. This will allow you to use the Sourcebot container in your NixOS deployment.
85+
86+
```nix wrap icon="code" Add flake input
87+
inputs.sourcebot.url = "github:sourcebot-dev/sourcebot";
88+
```
89+
90+
Add sourcebot module to your NixOS configuration:
91+
92+
```nix wrap icon="code" Add module to configuration
93+
nixosConfigurations.mysystem = nixpkgs.lib.nixosSystem {
94+
modules = [
95+
inputs.sourcebot.nixosModules.sourcebot
96+
];
97+
}
98+
```
99+
[Learn more about NixOS flakes](https://nixos.wiki/wiki/Flakes).
100+
</Step>
101+
<Step title="Setup Credentials">
102+
Sourcebot requires a few secrets to be set up before it can run, and code host credentials can be managed using NixOS module too:
103+
104+
- [sops-nix](https://github.com/Mic92/sops-nix) example:
105+
106+
```nix wrap icon="code" sops-nix configuration
107+
sops = {
108+
secrets = {
109+
sourcebot-auth-secret.owner = "sourcebot";
110+
sourcebot-encryption-key.owner = "sourcebot";
111+
sourcebot-gitlab-token.owner = "sourcebot";
112+
};
113+
templates = {
114+
sourcebot-env = {
115+
content = ''
116+
AUTH_SECRET=${config.sops.placeholder.sourcebot-auth-secret}
117+
SOURCEBOT_ENCRYPTION_KEY=${config.sops.placeholder.sourcebot-encryption-key}
118+
GITLAB_EXAMPLE_TOKEN=${config.sops.placeholder.sourcebot-gitlab-token}
119+
'';
120+
};
121+
};
122+
};
123+
```
124+
125+
- [agenix](https://github.com/ryantm/agenix) example:
126+
127+
```nix wrap icon="code" agenix configuration
128+
age.secrets.sourcebot-env.file = ../secrets/sourcebot.age;
129+
```
130+
131+
`sourcebot.age` file should be an environment file in the format:
132+
133+
```bash wrap icon="terminal" Environment file format
134+
AUTH_SECRET=your-auth-secret
135+
SOURCEBOT_ENCRYPTION_KEY=your-encryption-key
136+
GITLAB_EXAMPLE_TOKEN=your-gitlab-token
137+
```
138+
</Step>
139+
<Step title="Enable Sourcebot">
140+
The following NixOS configuration will enable Sourcebot and set it up to run with the provided configuration.
141+
Additional options could be found in the [source file](https://github.com/sourcebot-dev/sourcebot/blob/main/nix/nixosModule.nix).
142+
143+
```nix wrap icon="code" Enable Sourcebot service
144+
services.sourcebot = {
145+
enable = true;
146+
# envFile = config.sops.templates.sourcebot-env.path; # Uncomment if using sops-nix
147+
# envFile = config.age.secrets.sourcebot-env.path; # Uncomment if using agenix
148+
package = pkgs.sourcebot;
149+
logLevel = "info";
150+
dataDir = "/data/sourcebot";
151+
dataCacheDir = "/data/sourcebot/cache";
152+
configPath = "${pkgs.writeText "config" (builtins.toJSON {
153+
"$schema" = "https://raw.githubusercontent.com/sourcebot-dev/sourcebot/main/schemas/v3/index.json";
154+
connections = {
155+
github-public = {
156+
type = "github";
157+
repos = [
158+
"sourcebot-dev/sourcebot"
159+
];
160+
};
161+
gitlab-private = {
162+
type = "gitlab";
163+
url = "https://gitlab.example.com";
164+
all = true;
165+
token = {
166+
env = "GITLAB_EXAMPLE_TOKEN";
167+
};
168+
exclude = {
169+
forks = true;
170+
};
171+
};
172+
};
173+
settings = {
174+
resyncConnectionIntervalMs = 1000 * 60 * 60 * 24 * 7; # 1 week
175+
reindexIntervalMs = 1000 * 60 * 60 * 24 * 7; # 1 week
176+
maxRepoIndexingJobConcurrency = 1000; # 8 default
177+
maxConnectionSyncJobConcurrency = 1000; # 8 default
178+
maxRepoGarbageCollectionJobConcurrency = 1000; # 8 default
179+
};
180+
})}";
181+
};
182+
```
183+
</Step>
184+
185+
<Step title="Complete onboarding">
186+
Navigate to `http://localhost:7734` and complete the onboarding flow.
187+
</Step>
188+
189+
<Step title="Done">
190+
You're all set! If you'd like to setup [Ask Sourcebot](/docs/features/ask/overview), configure a language model [provider](/docs/configuration/language-model-providers).
191+
</Step>
192+
</Steps>
193+
194+
195+
196+
## Next steps
197+
---
198+
199+
<CardGroup cols={3}>
200+
<Card title="Index your code" icon="code" href="/docs/connections/overview">
201+
Learn how to index your code using Sourcebot
202+
</Card>
203+
<Card title="Language models" icon="brain" href="/docs/configuration/language-model-providers">
204+
Learn how to configure language model providers to start using [Ask Sourcebot](/docs/features/ask/overview)
205+
</Card>
206+
<Card title="Authentication" icon="lock" href="/docs/configuration/auth/overview">
207+
Learn more about how to setup SSO, email codes, and other authentication providers.
208+
</Card>
209+
</CardGroup>

‎flake.lock‎

Lines changed: 133 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)