Skip to content

GitLab

  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • slapos slapos
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Merge requests 122
    • Merge requests 122
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Jobs
  • Commits
Collapse sidebar
  • nexedi
  • slaposslapos
  • Merge requests
  • !1672

Merged
Created Oct 24, 2024 by Kirill Smelkov@kirrOwner

software/ors-amarisoft: Fix fluentbit ingestion not to barf on large lines

  • Overview 5
  • Commits 1
  • Changes 2

Fluentbit Tail input documentation[1] says that by default maximum buffer size is 32K which turned out to be too small in practice because we hit a situation where enb.xlog started to have lines with ~ 34K and so fluentbit ingestion stopped to work with the following error in fluentbit log:

[2024/10/23 20:30:23] [error] [input:tail:tail.0] file=/srv/slapgrid/slappart19/srv/monitor/public/enb.xlog requires a larger buffer size, lines are too long. Skipping file.

-> Fix that by increasing max buffer size to 1M which seems to be high enough at least for now.

Maybe it will make sense to configure this as unlimited, but I'm not sure if going as unlimited is universally a good idea.

[1] https://docs.fluentbit.io/manual/pipeline/inputs/tail

/cc @lu.xu, @jhuge, @tomo, @paul.graydon

Assignee
Assign to
Reviewer
Request review from
Time tracking
Source branch: y/fluent-1M
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7