Commit Graph

160 Commits

Author SHA1 Message Date
Robert Schäfer
146bf65d02 refactor(dev): remove ulimits for rootless docker (#24393)
Description
-----------

When I follow the [developer setup](https://docs.immich.app/developer/setup) I run into a permission error using rootless docker. A while ago I asked on Discord in [#contributing](https://discord.com/channels/979116623879368755/1071165397228855327/1442974448776122592) about these ulimits.

I suggest to remove the `ulimits` altogether. It seems that @ItalyPaleAle has left the setting just hoping that it could help somebody in the future. See the [PR description](https://github.com/immich-app/immich/pull/4556).

How Has This Been Tested?
-------------------------

Using rootless docker:

```
$ docker context ls
NAME         DESCRIPTION                               DOCKER ENDPOINT                     ERROR
default                                                unix:///var/run/docker.sock
rootless *                                             unix:///run/user/1000/docker.sock
```

Running `make` will fail because of permission errors:
```
$  docker compose -f ./docker/docker-compose.dev.yml up --remove-orphans
...
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting rlimits for ready process: error setting rlimit type 7: operation not permitted
```

On my machine I have the following hard limit for "Maximum number of open file descriptors":
```
$ ulimit -nH
524288
```

I can confirm that the permission error is caused by the security restrictions of the operating system mentioned above:

Changing `docker/docker-compose.dev.yml` like ..

```
    ulimits:
      nofile:
        soft: 524289
        hard: 524289
```

.. will lead to a permission error whereas this ..

```
    ulimits:
      nofile:
        soft: 524288
        hard: 524288
```

.. starts fine.

Apparently the defaults for these limits are coming from [systemd](26b2085d54/man/systemd.exec.xml (L1122)) which is used on nearly every linux distribution. So my assumption is that almost any linux user who uses rootless docker will run into a permission error when starting the development setup.

Checklist:
----------

- [x] I have performed a self-review of my own code
- [x] I have made corresponding changes to the documentation if applicable
- [x] I have no unrelated changes in the PR.
- [ ] I have confirmed that any new dependencies are strictly necessary.
- [ ] I have written tests for new code (if applicable)
- [ ] I have followed naming conventions/patterns in the surrounding code
- [ ] All code in `src/services/` uses repositories implementations for database calls, filesystem operations, etc.
- [ ] All code in `src/repositories/` is pretty basic/simple and does not have any immich specific logic (that belongs in `src/services/`)
2025-12-05 09:26:20 -05:00
renovate[bot]
bb341cc774 chore(deps): update docker.io/valkey/valkey docker tag to v9 (#24336)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-12-02 09:26:39 -05:00
Alex
4dcc049465 feat: workflow foundation (#23621)
* feat: plugins

* feat: table definition

* feat: type and migration

* feat: add repositories

* feat: validate manifest with class-validator and load manifest info to database

* feat: workflow/plugin controller/service layer

* feat: implement workflow logic

* feat: make trigger static

* feat: dynamical instantiate plugin instances

* fix: access control and helper script

* feat: it works

* chore: simplify

* refactor: refactor and use queue for workflow execution

* refactor: remove unsused property in plugin-schema

* build wasm in prod

* feat: plugin loader in transaction

* fix: docker build arm64

* generated files

* shell check

* fix tests

* fix: waiting for migration to finish before loading plugin

* remove context reassignment

* feat: use mise to manage extism tools (#23760)

* pr feedback

* refactor: create workflow now including create filters and actions

* feat: workflow medium tests

* fix: broken medium test

* feat: medium tests

* chore: unify workflow job

* sign user id with jwt

* chore: query plugin with filters and action

* chore: read manifest in repository

* chore: load manifest from server configs

* merge main

* feat: endpoint documentation

* pr feedback

* load plugin from absolute path

* refactor:handle trigger

* throw error and return early

* pr feedback

* unify plugin services

* fix: plugins code

* clean up

* remove triggerConfig

* clean up

* displayName and methodName

---------

Co-authored-by: Jason Rasmussen <jason@rasm.me>
Co-authored-by: bo0tzz <git@bo0tzz.me>
2025-11-14 20:05:05 +00:00
Kang
02b29046b3 feat: ocr (#18836)
* feat: add OCR functionality and related configurations

* chore: update labeler configuration for machine learning files

* feat(i18n): enhance OCR model descriptions and add orientation classification and unwarping features

* chore: update Dockerfile to include ccache for improved build performance

* feat(ocr): enhance OCR model configuration with orientation classification and unwarping options, update PaddleOCR integration, and improve response structure

* refactor(ocr): remove OCR_CLEANUP job from enum and type definitions

* refactor(ocr): remove obsolete OCR entity and migration files, and update asset job status and schema to accommodate new OCR table structure

* refactor(ocr): update OCR schema and response structure to use individual coordinates instead of bounding box, and adjust related service and repository files

* feat: enhance OCR configuration and functionality

- Updated OCR settings to include minimum detection box score, minimum detection score, and minimum recognition score.
- Refactored PaddleOCRecognizer to utilize new scoring parameters.
- Introduced new database tables for asset OCR data and search functionality.
- Modified related services and repositories to support the new OCR features.
- Updated translations for improved clarity in settings UI.

* sql changes

* use rapidocr

* change dto

* update web

* update lock

* update api

* store positions as normalized floats

* match column order in db

* update admin ui settings descriptions

fix max resolution key

set min threshold to 0.1

fix bind

* apply config correctly, adjust defaults

* unnecessary model type

* unnecessary sources

* fix(ocr): switch RapidOCR lang type from LangDet to LangRec

* fix(ocr): expose lang_type (LangRec.CH) and font_path on OcrOptions for RapidOCR

* fix(ocr): make OCR text search case- and accent-insensitive using ILIKE + unaccent

* fix(ocr): add OCR search fields

* fix: Add OCR database migration and update ML prediction logic.

* trigrams are already case insensitive

* add tests

* format

* update migrations

* wrong uuid function

* linting

* maybe fix medium tests

* formatting

* fix weblate check

* openapi

* sql

* minor fixes

* maybe fix medium tests part 2

* passing medium tests

* format web

* readd sql

* format dart

* disabled in e2e

* chore: translation ordering

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2025-10-27 14:09:55 +00:00
Matthew Momjian
74a9be4a0e fix(server): bump valkey to 8 (#22911)
* unpin valkey

* pin
2025-10-15 18:48:36 -04:00
renovate[bot]
e8ca7f235c chore(deps): update ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0 docker digest to bcf6335 (#22913)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-10-13 21:14:33 -05:00
gablilli
1086623457 chore: replace immich.app/docs with docs.immich.app globally (#22428)
* fix: es

* fix(readme): fix docs link

* fix(readme): fix main readme's docs link

* fix: main readme

* fix: replace immich.app/docs with docs.immich.app globally
2025-10-02 10:42:14 -05:00
renovate[bot]
8a3684c127 chore(deps): update ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0 docker digest to 41eacbe (#22305)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-09-26 14:26:55 +02:00
Jason Rasmussen
b9e2590752 chore: simplify (#22082) 2025-09-16 12:48:44 -04:00
Jason Rasmussen
1d6b98ff86 chore: remove prepare-volumes (#22071) 2025-09-16 10:19:09 -04:00
Jason Rasmussen
5fad1fd899 chore: split Dockerfile (#22007) 2025-09-15 23:41:35 -04:00
renovate[bot]
5fb0afb0d0 chore(deps): update ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0 docker digest to c44be5f (#22038)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-09-15 23:13:21 -04:00
Min Idzelis
a10a946d1a fix: let dev docker compose service runs as root (#21579) 2025-09-12 16:20:41 +01:00
renovate[bot]
ee98e69097 chore(deps): update docker.io/valkey/valkey:8-bookworm docker digest to fea8b3e (#21281)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-09-10 08:36:44 -04:00
renovate[bot]
41c1d4d44b chore(deps): update ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0 docker digest to 8d292bd (#21287)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-09-10 08:36:27 -04:00
Min Idzelis
147accd957 fix: fix docker perms for dev (#21359) 2025-08-28 22:07:29 -04:00
renovate[bot]
1298a74230 chore(deps): pin busybox docker tag to ab33eac (#21280)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-08-27 21:44:19 -05:00
Jason Rasmussen
d04675fb41 fix: dev-scripts (#21270)
fix/dev-scripts
2025-08-25 17:10:59 -04:00
Min Idzelis
44e1c83c84 fix: isolate docker host/container filesystem for node_modules and build output (#21167) 2025-08-24 13:09:45 -05:00
Jason Rasmussen
66c657ca8a chore: remove web/Dockerfile (#21096) 2025-08-21 12:55:23 -04:00
renovate[bot]
f08002d48f chore(deps): update docker.io/valkey/valkey:8-bookworm docker digest to a137a2b (#21043)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-08-19 09:13:24 -04:00
Min Idzelis
845b0f2073 chore: use pnpm for builds (#19752)
* Migrate from npm to pnpm across entire project

• Update all GitHub workflow files to use pnpm instead of npm
• Replace npm commands with pnpm equivalents in devcontainer scripts
• Remove package-lock.json files and update to use pnpm-lock.yaml
• Consolidate node version references to use server/.nvmrc

* Refine pnpm migration based on review feedback

• Replace SKIP_SHARP_FILTERING with SHARP_IGNORE_GLOBAL_LIBVIPS environment variable
• Improve Sharp package filtering to include specific Linux architectures for Docker builds
• Optimize Dockerfile dependency caching with improved layer structure
• Clean up workspace configuration and remove redundant settings

* Address additional review feedback for pnpm migration

• Fix node-version-file paths in GitHub workflow configurations
• Refactor .pnpmfile.cjs to use switch statement for better code organization
• Correct cache type typo in fix-format workflow
• Simplify Vite configuration by merging configs inline
• Update package description for consistency

* Use 'server/.nvmrc' for fix-format.yml GHA

* Delete npm locks

* Remove Docker volume isolation for node_modules directories

• Remove volume mounts for node_modules and related directories
• Allow shared access between host and container filesystem
• Update init container to handle file ownership with conditional existence check

* Remove unused Docker volumes and volume mounts

• Remove node_modules volume mounts from devcontainer configuration
• Remove unused named volumes for pnpm-store, node_modules, and cache directories
• Clean up Docker Compose configuration after removing volume isolation

* Fix typescript-sdk package issues

• Remove unknown "build" dependency that was incorrectly added to package.json
• Update pnpm-lock.yaml to reflect dependency removal

* Add pnpm setup to mobile workflow for translation formatting

• Add pnpm action setup step to mobile unit tests workflow
• Required for translation file formatting and sorting operations

---------

Co-authored-by: Jason Rasmussen <jason@rasm.me>
2025-08-19 08:55:24 -04:00
renovate[bot]
53f9fc2d1c chore(deps): update docker.io/valkey/valkey:8-bookworm docker digest to 5b8f8c3 (#20874)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-08-13 21:49:38 +02:00
renovate[bot]
d8a6552811 chore(deps): update ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0 docker digest to 32324a2 (#20381)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-07-30 08:29:37 -04:00
Jason Rasmussen
58521c9efb feat: change default media location to /data (#20367)
* feat!: change default media location to /data

* feat: dynamically detect media location
2025-07-29 16:58:50 -04:00
Jason Rasmussen
c6b25ef111 feat: automatically detect media location changes (#20256) 2025-07-25 15:25:36 -04:00
Min Idzelis
9e94f52b05 chore: dockerfile layout changes (#19673)
Dockerfile layout changes

Fix up web path

feat: update server env vars for layout
2025-07-18 17:56:26 -04:00
Min Idzelis
f33e1ad94c feat: relocate scripts, PATH update (#20002)
Relocate scripts, and PATH updates
2025-07-18 11:19:06 -04:00
renovate[bot]
eca54871d0 chore(deps): update docker.io/valkey/valkey:8-bookworm docker digest to facc1d2 (#19932)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-07-15 15:43:36 +00:00
Matthew Momjian
2a8019726c feat(deployment): add shm_size for PG (#19472) 2025-06-25 20:38:37 +01:00
renovate[bot]
6c7b2e4b5c chore(deps): pin ghcr.io/immich-app/postgres docker tag to 5f6a838 (#19480)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-06-24 11:19:40 +01:00
Mert
7c43e6c3c8 fix: bump vchord default to 0.4.3 (#19365)
bump default to 0.4.3
2025-06-20 08:35:32 -05:00
Alex
75bf3aa1be chore: correct vchord version in docker-compose.yml (#19251) 2025-06-19 09:11:15 -05:00
renovate[bot]
00536bf074 chore(deps): update docker.io/valkey/valkey:8-bookworm docker digest to fec42f3 (#19218)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-06-17 11:57:42 +01:00
renovate[bot]
a9bd651692 chore(deps): update docker.io/valkey/valkey:8-bookworm docker digest to a19bebe (#18879)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-06-05 10:49:30 +01:00
Mert
3f08768854 chore: vchord 0.4.1 (#18588)
* vchord 0.4.x

* oops

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2025-05-28 14:38:52 +00:00
Daimolean
6abcfaef99 docs: update link (#18689) 2025-05-27 16:22:57 +00:00
Alex
f6903696cb fix: revert accidental docker-compose dev change (#18686) 2025-05-27 17:15:45 +01:00
Alex
fff651f8a5 fix(web): handle nullable assets duration (#18679)
* fix(web): handle nullable assets duration

* Update web/src/lib/components/assets/thumbnail/thumbnail.svelte

Co-authored-by: Mert <101130780+mertalev@users.noreply.github.com>

* fix: format

---------

Co-authored-by: Mert <101130780+mertalev@users.noreply.github.com>
2025-05-27 10:24:17 -05:00
renovate[bot]
09d9fa9755 chore(deps): pin dependencies (#18659)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-05-27 12:10:30 +02:00
Mert
0d773af6c3 feat: vectorchord (#18042)
* wip

auto-detect available extensions

auto-recovery, fix reindexing check

use original image for ml

* set probes

* update image for sql checker

update images for gha

* cascade

* fix new instance

* accurate dummy vector

* simplify dummy

* preexisiting pg docs

* handle different db name

* maybe fix sql generation

* revert refreshfaces sql change

* redundant switch

* outdated message

* update docker compose files

* Update docs/docs/administration/postgres-standalone.md

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* tighten range

* avoid always printing "vector reindexing complete"

* remove nesting

* use new images

* add vchord to unit tests

* debug e2e image

* mention 1.107.2 in startup error

* support new vchord versions

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
2025-05-20 08:36:43 -05:00
renovate[bot]
60c43081ed chore(deps): update docker.io/valkey/valkey:8-bookworm docker digest to ff21bc0 (#18245)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-05-12 22:25:56 -04:00
renovate[bot]
b8509e6411 chore(deps): update docker.io/valkey/valkey:8-bookworm docker digest to 4a9f847 (#18113)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-05-06 08:35:24 -04:00
renovate[bot]
9851d24628 chore(deps): update docker.io/valkey/valkey:8-bookworm docker digest to c855f98 (#17948)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-29 12:08:50 +01:00
Mert
7b6a4be30c chore: use valkey (#17396)
use valkey
2025-04-04 17:46:46 -05:00
Mert
2b37caba03 feat(ml): rocm (#16613)
* feat(ml): introduce support of onnxruntime-rocm for AMD GPU

* try mutex for algo cache

use OrtMutex

* bump versions, run on mich

use 3.12

use 1.19.2

* acquire lock before any changes can be made

guard algo benchmark results

mark mutex as mutable

re-add /bin/sh (?)

use 3.10

use 6.1.2

* use composite cache key

1.19.2

fix variable name

fix variable reference

aaaaaaaaaaaaaaaaaaaa

* bump deps

* disable algo caching

* fix gha

* try ubuntu runner

* actually fix the gha

* update patch

* skip mimalloc preload for rocm

* increase build threads

* increase timeout for rocm

* Revert "increase timeout for rocm"

This reverts commit 2c4452f5d132198ed381a7b262b4a5cab5114b5f.

* attempt migraphx

* set migraphx_home

* Revert "set migraphx_home"

This reverts commit c121d3e48754b3bce100636f8d666deec58a44b7.

* Revert "attempt migraphx"

This reverts commit 521f9fb72dbe506dc6cb8faeb6494817d87265c6.

* migraphx, take two

* bump rocm

* allow cpu

* try only targeting migraphx

* skip tests

* migraph 

* known issues

* target gfx900 and gfx1102

* mention `HSA_USE_SVM`

* update lock

* set device id for rocm

---------

Co-authored-by: Mehdi GHESH <mehdi.ghesh@hotmail.fr>
2025-03-17 21:08:19 +00:00
Yoni Yang
14c3b99c0f feat(ml): ML on Rockchip NPUs (#15241) 2025-03-17 12:04:08 -04:00
Nicholas Flamy
c6ede48e59 fix(server): set the dev server restart policy of the dev server container to match the other containers (#16753)
set the restart policy of the dev server container to match the other containers
2025-03-09 22:25:03 -05:00
renovate[bot]
9b842d4cca chore(deps): update tensorchord/pgvecto-rs:pg14-v0.2.0 docker digest to 739cdd6 (#16530)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-03-03 12:13:38 +00:00
renovate[bot]
b3fbd0809b chore(deps): update redis:6.2-alpine docker digest to 148bb54 (#16140)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-17 14:23:03 +00:00