Commit graph

97 commits

Author SHA1 Message Date
crapStone
a9b6615286 create a workaround to fix the oom problem 2024-03-15 20:38:59 +01:00
Hoernschen
a6e9510c07 FIX blank internal pages (#164) (#292)
Hello 👋

since it affected my deployment of the pages server I started to look into the problem of the blank pages and think I found a solution for it:

1. There is no check if the file response is empty, neither in cache retrieval nor in writing of a cache. Also the provided method for checking for empty responses had a bug.
2. I identified the redirect response to be the issue here. There is a cache write with the full cache key (e. g. rawContent/user/repo|branch|route/index.html) happening in the handling of the redirect response. But the written body here is empty. In the triggered request from the redirect response the server then finds a cache item to the key and serves the empty body. A quick fix is the check for empty file responses mentioned in 1.
3. The decision to redirect the user comes quite far down in the upstream function. Before that happens a lot of stuff that may not be important since after the redirect response comes a new request anyway. Also, I suspect that this causes the caching problem because there is a request to the forge server and its error handling with some recursions happening before. I propose to move two of the redirects before "Preparing"
4. The recursion in the upstream function makes it difficult to understand what is actually happening. I added some more logging to have an easier time with that.
5. I changed the default behaviour to append a trailing slash to the path to true. In my tested scenarios it happened anyway. This way there is no recursion happening before the redirect.

I am not developing in go frequently and rarely contribute to open source -> so feedback of all kind is appreciated

closes #164

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/292
Reviewed-by: 6543 <6543@obermui.de>
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: Hoernschen <julian.hoernschemeyer@mailbox.org>
Co-committed-by: Hoernschen <julian.hoernschemeyer@mailbox.org>
2024-02-26 22:21:42 +00:00
crapStone
7e80ade24b Add config file and rework cli parsing and passing of config values (#263)
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/263
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: crapStone <me@crapstone.dev>
Co-committed-by: crapStone <me@crapstone.dev>
2024-02-15 16:08:29 +00:00
crapStone
c1fbe861fe rename gitea to forge in html error messages (#287)
closes #286

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/287
Reviewed-by: Andreas Shimokawa <ashimokawa@noreply.codeberg.org>
Co-authored-by: crapStone <crapstone01@gmail.com>
Co-committed-by: crapStone <crapstone01@gmail.com>
2024-02-11 12:43:25 +00:00
Gusted
15916444e1 Fix panic in formatting not found message (#276)
Fix panic that was found in the logs, `targetOpt` is `nil`:

http2: panic serving 10.0.3.1:[...]: runtime error: invalid memory address or nil pointer dereference
net/http.(*http2serverConn).runHandler.func1()
        /usr/local/go/src/net/http/h2_bundle.go:6104 +0x145
panic({0x19c6820, 0x2d66db0})
        /usr/local/go/src/runtime/panic.go:884 +0x213
codeberg.org/codeberg/pages/server/handler.handleSubDomain({{0x2008c68, 0xc00047df90}, 0x2, {0x0, 0x0}, {0xc0fe3ef800, 0x55, 0x1f4}, {0xc00047dfa0, 0x1, ...}, ...}, ...)
        /woodpecker/src/codeberg.org/Codeberg/pages-server/server/handler/handler_sub_domain.go:59 +0x5e0

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/276
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
2024-01-18 14:31:46 +00:00
crapStone
fffb8ffcb6 remove use of rawInfoPage redirect (#261)
closes #244

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/261
Co-authored-by: crapStone <crapstone01@gmail.com>
Co-committed-by: crapStone <crapstone01@gmail.com>
2023-11-16 17:33:39 +00:00
crapStone
cbb2ce6d07 add go templating engine for error page and make errors more clear (#260)
ping #199
closes #213

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/260
Co-authored-by: crapStone <crapstone01@gmail.com>
Co-committed-by: crapStone <crapstone01@gmail.com>
2023-11-16 17:11:35 +00:00
crapStone
1e1c67be93 let gitea client send user-agent with version (#258)
closes #255

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/258
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: crapStone <crapstone01@gmail.com>
Co-committed-by: crapStone <crapstone01@gmail.com>
2023-11-15 15:25:14 +00:00
crapStone
a8272f0ce9 Don't send server version to client (#254)
closes #247

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/254
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: crapStone <crapstone01@gmail.com>
Co-committed-by: crapStone <crapstone01@gmail.com>
2023-11-15 01:49:29 +00:00
Moritz Marquardt
56d3e291c4 Security Fix: clean paths correctly to avoid circumvention of BlacklistedPaths 2023-08-27 10:13:15 +02:00
6543
d720d25e42 Use http.NoBody as per linter (#231)
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/231
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2023-07-17 19:44:58 +00:00
Gusted
7f318f89a6 Fix escaped error message (#230)
- This specific message will [already be generated](https://codeberg.org/Codeberg/pages-server/src/commit/974229681f4cc7f1ed31df9b05eabef2df01380/html/error.go#L44) when `http.StatusMisdirectedRequest` is set as status with [an empty message](974229681f/html/error.go (L25-L28)).
- Resolves https://codeberg.org/Codeberg/pages-server/issues/228

Co-authored-by: Gusted <postmaster@gusted.xyz>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/230
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
2023-07-16 22:34:46 +00:00
video-prize-ranch
974229681f Initial redirects implementation (#148)
Adds basic support for `_redirects` files. It supports a subset of what IPFS supports: https://docs.ipfs.tech/how-to/websites-on-ipfs/redirects-and-custom-404s/

Example:
```
/redirect https://example.com/ 301
/another-redirect /page 301
/302  https://example.com/  302
/app/* /index.html 200
/articles/* /posts/:splat 301
```
301 redirect: https://video-prize-ranch.localhost.mock.directory:4430/redirect
SPA rewrite: https://video-prize-ranch.localhost.mock.directory:4430/app/path/path
Catch-all with splat: https://video-prize-ranch.localhost.mock.directory:4430/articles/path/path

Closes #46

Co-authored-by: video-prize-ranch <cb.8a3w5@simplelogin.co>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/148
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: video-prize-ranch <video-prize-ranch@noreply.codeberg.org>
Co-committed-by: video-prize-ranch <video-prize-ranch@noreply.codeberg.org>
2023-03-30 21:36:31 +00:00
Crystal
c40dddf471 Fix certificate renewal (#209)
A database bug in xorm.go prevents the pages-server from saving a
renewed certificate for a domain that already has one in the database.

Co-authored-by: crystal <crystal@noreply.codeberg.org>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/209
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Crystal <crystal@noreply.codeberg.org>
Co-committed-by: Crystal <crystal@noreply.codeberg.org>
2023-03-20 22:57:26 +00:00
6543
c9050e5722 Handle Relative Symlinks (#205)
enhance #114

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/205
2023-03-11 05:07:17 +00:00
deblan
42d5802b9b Allow to define default branches (#125)
This try to address #115

Co-authored-by: Simon Vieille <simon@deblan.fr>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/125
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: deblan <deblan@noreply.codeberg.org>
Co-committed-by: deblan <deblan@noreply.codeberg.org>
2023-02-14 03:03:00 +00:00
6543
0adac9a5b1 fix http -> https redirect and add integration tests for it (#184)
and more logging

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/184
2023-02-14 02:23:28 +00:00
6543
42b3f8d1b7 use mockery for mock code generation (#185)
close 	#181

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/185
2023-02-13 23:13:30 +00:00
6543
9a3d1c36dc Document more flags & make http port customizable (#183)
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/183
2023-02-13 20:14:45 +00:00
crystal
46316f9e2f Fix raw domain for branches with custom domains and index.html (#159)
fix #156
fix #157

Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/159
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: crystal <crystal@noreply.codeberg.org>
Co-committed-by: crystal <crystal@noreply.codeberg.org>
2023-02-11 03:12:42 +00:00
6543
5753f7136d Move acmeClient creation into own file & struct (#179)
get rid of gobal vars and make make functions with less args :)

tldr: collect funcs and create a own ACME client to manage that stuff

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/179
2023-02-11 02:29:08 +00:00
6543
fd643d15f0 Drop: pogreb support (#175)
followup of #173

close #95

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/175
2023-02-11 02:04:57 +00:00
6543
272c7ca76f Fix xorm regressions by handle wildcard certs correctly (#177)
close #176

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/177
2023-02-11 01:26:21 +00:00
6543
d8d119b0b3 Fix Cache Bug (#178)
error io.EOF is gracefully end of file read.

so we don't need to cancel cache saving

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/178
2023-02-11 00:31:56 +00:00
6543
7b35a192bf Add cert store option based on sqlite3, mysql & postgres (#173)
Deprecate **pogreb**!

close #169

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/173
2023-02-10 03:00:14 +00:00
foehammer
7fce7cf68b Added Whitespace Trimming TXT DNS Records (#152)
Solves https://codeberg.org/Codeberg/Community/issues/823 and https://codeberg.org/Codeberg/pages-server/issues/143

Co-authored-by: foehammer127 <foehammer127@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/152
Reviewed-by: Otto <otto@codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: foehammer <foehammer@noreply.codeberg.org>
Co-committed-by: foehammer <foehammer@noreply.codeberg.org>
2023-02-10 01:44:44 +00:00
Gusted
8b1f497bc4 Allow to use certificate even if domain validation fails (#160)
- Currently if the canonical domain validations fails(either for
legitimate reasons or for bug reasons like the request to Gitea/Forgejo
failing) it will use main domain certificate, which in the case for
custom domains will warrant a security error as the certificate isn't
issued to the custom domain.
- This patch handles this situation more gracefully and instead only
disallow obtaining a certificate if the domain validation fails, so in
the case that a certificate still exists it can still be used even if
the canonical domain validation fails. There's a small side effect,
legitimate users that remove domains from `.domain` will still be able
to use the removed domain(as long as the DNS records exists) as long as
the certificate currently hold by pages-server isn't expired.
- Given the increased usage in custom domains that are resulting in
errors, I think it ways more than the side effect.
- In order to future-proof against future slowdowns of instances, add a retry mechanism to the domain validation function, such that it's more likely to succeed even if the instance is not responding.
- Refactor the code a bit and add some comments.

Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/160
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
2023-02-10 01:38:15 +00:00
Gusted
513e79832a Use correct log level for CheckCanonicalDomain (#162)
- Currently any error generated by requesting the `.domains` file of a repository would be logged under the info log level, which isn't the correct log level when we exclude the not found error.
- Use warn log level if the error isn't the not found error.

Co-authored-by: Gusted <postmaster@gusted.xyz>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/162
Reviewed-by: Otto <otto@codeberg.org>
2023-01-22 18:52:21 +00:00
crapStone
bd538abd37 Fix wrong redirect on custom domain with path (#154)
closes #153

Co-authored-by: crapStone <crapstone01@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/154
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: crapStone <crapstone@noreply.codeberg.org>
Co-committed-by: crapStone <crapstone@noreply.codeberg.org>
2023-01-11 00:00:37 +00:00
Felipe Leopoldo Sologuren Gutiérrez
c286b3b1d0 Added TokenBucket to limit the rate of validation failures (#151)
Added new TockenBucket named `acmeClientFailLimit` to avoid being banned because of the [Failed validation limit](https://letsencrypt.org/docs/failed-validation-limit/) of Let's Encrypt.

The behaviour is similar to the other limiters blocking the `obtainCert` func ensuring rate under limit.

Co-authored-by: fsologureng <sologuren@estudiohum.cl>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/151
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Felipe Leopoldo Sologuren Gutiérrez <fsologureng@noreply.codeberg.org>
Co-committed-by: Felipe Leopoldo Sologuren Gutiérrez <fsologureng@noreply.codeberg.org>
2023-01-04 05:26:14 +00:00
Gusted
98d198d419 Safely get certificate's leaf (#150)
- It's not guaranteed that `tls.X509KeyPair` will set `c.Leaf`.
- This patch fixes this by using a wrapper that parses the leaf
certificate(in bytes) if `c.Leaf` wasn't set.
- Resolves #149

Co-authored-by: Gusted <postmaster@gusted.xyz>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/150
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
2023-01-04 04:51:27 +00:00
jklippel
caeb1a4acb Return a 404 if there is no repository (#141)
If no repository is found the user expects a 404 status code
instead of a dependency failed status code (as it was before).

Signed-off-by: Jan Klippel <c0d3b3rg@kl1pp3l.de>

Fixes: https://codeberg.org/Codeberg/Community/issues/809

Co-authored-by: Jan Klippel <c0d3b3rg@kl1pp3l.de>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/141
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: jklippel <jklippel@noreply.codeberg.org>
Co-committed-by: jklippel <jklippel@noreply.codeberg.org>
2022-11-22 21:26:10 +00:00
Gusted
f2f943c0d8 Remove unnecessary conversion (#139)
- Remove unnecessary type conversion.
- Enforce via CI

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/139
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
2022-11-15 16:15:11 +01:00
Gusted
3c61a39864 Enable http/2 support (#137)
As per [the documentation](https://pkg.go.dev/net/http#Serve), it doesn't enable HTTP2 by-default, unless we enable it via the `NextProtos` option.

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/137
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
2022-11-12 22:25:20 +01:00
6543
4565481643
refactor: finish use default const for defaultPagesBranch and defaultPagesRepo 2022-11-12 21:16:11 +01:00
6543
c827a28dd8
defaultPagesBranch 2022-11-12 21:13:13 +01:00
6543
aa90356f0a
use a const for defaultPagesRepo 2022-11-12 21:10:16 +01:00
6543
b6d0a04b21
refactor: rm not needed type conversion 2022-11-12 21:04:34 +01:00
6543
6c63b66ce4 Refactor split long functions (#135)
we have big functions that handle all stuff ... we should split this into smaler chuncks so we could test them seperate and make clear cuts in what happens where

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/135
2022-11-12 20:43:44 +01:00
6543
b9966487f6 switch to std http implementation instead of fasthttp (#106)
close #100
close #109
close #113
close #28
close #63

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/106
2022-11-12 20:37:20 +01:00
6543
91b54bef29
add newline 2022-11-07 23:09:41 +01:00
Gusted
bf9a08e1fd Fatal on ACME Client creation failure (#133)
- For production(*cough* Codeberg *cough*), it's important to not use
mock certs. So fail right from the start if this is the case and not try
to "handle it gracefully", as it would break production.
- Resolves #131

CC @6543

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/133
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
2022-11-07 16:27:37 +01:00
Gusted
091e6c8ed9 Add explicit logging in GetBranchTimestamp (#130)
- Logs are currently indicating that it's returning `nil` in valid
scenarios, therefor this patch adds extra logging in this code to
better understand what it is doing in this function.

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/130
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
2022-09-18 16:13:27 +02:00
Gusted
8f2699407d Make verbose checks in tryBranch (#127)
- It's likely that the tryBranch is returning false when it should be returning true, make these logs more verbose so they show up on production logs.

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/127
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
2022-09-13 23:06:31 +02:00
Gusted
1ae50735a1 Add host to handler logging (#123)
- Add the host to the Handler's logging fields, so you don't just see the path, but also which domain was being requested.

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/123
Reviewed-by: 6543 <6543@noreply.codeberg.org>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
2022-08-13 18:03:31 +02:00
6543
dc41a4caf4 Add Support to Follow Symlinks and LFS (#114)
close #79
close #80
close #91

Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/114
2022-08-12 06:40:12 +02:00
Gusted
876a53d9a2 Improve logging (#116)
- Actually log useful information at their respective log level.
- Add logs in hot-paths to be able to deep-dive and debug specific requests (see server/handler.go)
- Add more information to existing fields(e.g. the host that the user is visiting, this was noted by @fnetX).

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/116
Reviewed-by: 6543 <6543@noreply.codeberg.org>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
2022-08-12 05:06:26 +02:00
6543
e06900d5e5
fix lint issue 2022-08-08 15:25:31 +02:00
6543
8207586a48
just fix bcaceda711 2022-07-15 21:39:42 +02:00
6543
bcaceda711 dont cache if ContentLength greater fileCacheSizeLimit (#108)
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/108
Reviewed-by: Otto <otto@codeberg.org>
2022-07-15 21:21:26 +02:00