Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: add immutable option #126

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

Warxcell
Copy link

@Warxcell Warxcell commented Aug 9, 2024

Reason for This PR

Fixes roadrunner-server/roadrunner#1985

Description of Changes

Adds immutable option to static HTTP middleware.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under the terms of the MIT license.

PR Checklist

[Author TODO: Meet these criteria.]
[Reviewer TODO: Verify that these criteria are met. Request changes if not]

  • All commits in this PR are signed (git commit -s).
  • The reason for this PR is clearly provided (issue no. or explanation).
  • The description of changes is clear and encompassing.
  • Any required documentation changes (code and docs) are included in this PR.
  • Any user-facing changes are mentioned in CHANGELOG.md.
  • All added/changed functionality is tested.

Summary by CodeRabbit

  • New Features

    • Introduced an Immutable configuration option for enhanced control over setting mutability.
    • Enhanced file-serving capabilities in the application, allowing for improved error handling and configurable behavior when serving static files.
  • Bug Fixes

    • Fixed issues related to file existence and directory checks during file serving.
  • Refactor

    • Improved modularity by separating the calculation and setting of ETags for better maintainability.

@Warxcell Warxcell marked this pull request as draft August 9, 2024 10:43
Copy link

coderabbitai bot commented Aug 9, 2024

Walkthrough

The changes introduce an Immutable field in the Config struct, enhancing configuration control by allowing settings to be treated as immutable. Additionally, the Plugin struct's file-serving capabilities are significantly improved with new functions for handling static files, including a streamlined server that supports immutable file serving. These updates aim to optimize performance and control over file handling in applications, particularly in environments with read-only filesystems.

Changes

Files Change Summary
config.go Added an Immutable boolean field to the Config struct for enhanced configuration control.
etag.go Refactored the SetEtag function into calculateEtag for ETag computation and a modified SetEtag for header setting, improving modularity.
plugin.go Introduced a FileServer function type and implemented createMutableServer and createImmutableServer functions to enhance file serving logic and error handling.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Plugin
    participant Server

    User->>Plugin: Request File
    Plugin->>Server: Serve File
    Server->>Plugin: Check File Existence
    Plugin->>Server: Return File Content
    Server->>User: Send File Response
Loading

Assessment against linked issues

Objective Addressed Explanation
Static HTTP middleware should support immutable option. (#1985)
Middleware should not slow down request processing. (#1985)
Implement caching properties of files at startup. (#1985) Caching logic for file properties not implemented.

Poem

Hop along, oh joyful day,
Immutable paths now light the way.
With files that stay as they should be,
Our Plugin sings, "Come serve with glee!"
In rabbit's heart, a leap of cheer,
For changes bright, the time is near! 🐇✨


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (1)
plugin.go (1)

242-246: Testing for Middleware function is missing.

The repository lacks test files, indicating that the Middleware function's dynamic server selection logic is not currently tested. It's important to add tests to ensure that the function behaves correctly under different configurations.

  • Consider creating test cases for both mutable and immutable configurations to verify the server selection logic.
Analysis chain

Dynamic server selection in Middleware is well-implemented.

The Middleware method correctly selects the server function based on the Immutable configuration.

Ensure that this logic is tested and behaves as expected.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the testing of dynamic server selection in `Middleware`.

# Test: Search for test cases that cover both mutable and immutable configurations.
rg --type go 'Middleware' -A 20

Length of output: 1105


Script:

#!/bin/bash
# Search for Go test files in the repository to find potential tests for the Middleware function.
fd --type f --extension go --exec rg 'func Test' {}

Length of output: 55


Script:

#!/bin/bash
# Search for all Go files in the repository to identify any potential test files.
fd --type f --extension go

Length of output: 56

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 1016760 and 266aa51.

Files selected for processing (2)
  • config.go (1 hunks)
  • plugin.go (3 hunks)
Additional comments not posted (3)
config.go (1)

21-22: Addition of Immutable field looks good.

The addition of the Immutable field to the Config struct is consistent with the existing configuration structure.

Ensure that this field is utilized correctly in the rest of the codebase.

Verification successful

Immutable Field Usage Verified

The Immutable field in the Config struct is used correctly in the codebase. It is utilized in plugin.go to conditionally create an immutable server, aligning with its intended purpose. No other usages were found, indicating focused and appropriate application.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of the `Immutable` field in the codebase.

# Test: Search for the usage of the `Immutable` field. Expect: Occurrences in plugin.go and other relevant files.
rg --type go 'Immutable'

Length of output: 228


Script:

#!/bin/bash
# Search for all occurrences of the `Immutable` field in Go files to ensure it is used correctly across the codebase.
rg --type go 'Immutable'

Length of output: 228

plugin.go (2)

37-38: Addition of FileServer type is appropriate.

The FileServer type provides a flexible abstraction for implementing file-serving logic.


98-158: Clarify immutable logic in createImmutableServer.

The createImmutableServer function should explicitly handle immutable logic, such as caching file properties. Ensure that this logic is implemented.

plugin.go Outdated Show resolved Hide resolved
@Warxcell Warxcell marked this pull request as ready for review August 9, 2024 14:50
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (1)
etag.go (1)

Line range hint 17-46:
Consider improving error handling in calculateEtag.

Currently, the function returns an empty ETag if an error occurs while reading the file. Consider logging the error or providing a default ETag to ensure consistent behavior.

// Example: Log error and return a default ETag
if err != nil {
    // Log the error or handle it appropriately
    return defaultEtag
}
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 266aa51 and 698b924.

Files selected for processing (2)
  • etag.go (2 hunks)
  • plugin.go (4 hunks)
Additional comments not posted (5)
etag.go (2)

Line range hint 17-46:
ETag calculation logic is sound.

The logic for calculating ETag using CRC32 checksum is correctly implemented and efficient.


49-51: LGTM! The SetEtag function is correctly implemented.

The function sets the ETag header in the HTTP response as expected.

plugin.go (3)

40-41: LGTM! The FileServer type enhances modularity.

The introduction of the FileServer function type allows for flexible and modular server implementations.


42-99: Consider improving error handling in server.

The current error handling logs all errors as "no such file or directory." Consider distinguishing between different error types to provide more accurate logging.


Line range hint 259-321:
LGTM! The Middleware function is well-implemented.

The function correctly selects between the standard and immutable servers based on configuration.

plugin.go Outdated Show resolved Hide resolved
@rustatian rustatian self-requested a review August 9, 2024 15:21
@rustatian rustatian added the enhancement New feature or request label Aug 9, 2024
@rustatian rustatian changed the title add immutable option feature: add immutable option Aug 9, 2024
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (1)
plugin.go (1)

Line range hint 327-343:
Document the use of unsafe operations.

The use of the unsafe package for converting between byte slices and strings should be documented to indicate potential risks and justify its use for performance.

// bytesToStr converts a byte slice to a string using unsafe operations for performance.
// Note: This bypasses usual safety checks and should be used with caution.
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 698b924 and 20bbe59.

Files selected for processing (1)
  • plugin.go (4 hunks)
Additional comments not posted (4)
plugin.go (4)

42-99: Consider improving error handling in server function.

The current error handling logs all errors as "no such file or directory." Consider distinguishing between different error types to provide more accurate logging.

if os.IsNotExist(err) {
	s.log.Debug("file does not exist", zap.Error(err))
} else {
	s.log.Error("error opening file", zap.Error(err))
}

108-178: Avoid using panic for error handling in createImmutableServer.

Using panic for error handling is not recommended in production code. Consider returning an error or logging it instead.

if err != nil {
    s.log.Error("error walking the path", zap.Error(err))
    return nil
}

Line range hint 180-260:
Initialization logic looks good.

The Init method properly configures the plugin and initializes necessary components.


Line range hint 262-325:
Middleware logic is correctly implemented.

The method correctly selects between the standard and immutable server based on configuration.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 20bbe59 and 3fc3c06.

Files selected for processing (1)
  • plugin.go (4 hunks)
Additional comments not posted (3)
plugin.go (3)

42-99: Consider improving error handling in server function.

The current error handling logs all errors as "no such file or directory." Consider distinguishing between different error types to provide more accurate logging.

if os.IsNotExist(err) {
	s.log.Debug("file does not exist", zap.Error(err))
} else {
	s.log.Error("error opening file", zap.Error(err))
}

108-179: Avoid using panic for error handling in createImmutableServer.

Using panic for error handling is not recommended in production code. Consider returning an error or logging it instead.

if err != nil {
    s.log.Error("error walking the path", zap.Error(err))
    return nil, err
}

330-330: LGTM!

The invocation of the server function is correctly implemented and aligns with the configuration logic.

plugin.go Outdated
Comment on lines 262 to 273
func (s *Plugin) Middleware(next http.Handler) http.Handler {
var server FileServer = server

if s.cfg.Immutable {
immutableServer, err := createImmutableServer(s)
if err != nil {
panic(err)
}

server = immutableServer
}

Copy link

@coderabbitai coderabbitai bot Aug 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider handling errors without using panic in Middleware.

Using panic during startup is understandable, but returning an error might improve maintainability and clarity. It allows for more controlled error handling and can provide additional context or perform cleanup if necessary.

if err != nil {
    s.log.Error("failed to create immutable server", zap.Error(err))
    return nil
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should have said that these errors can be returned from Init :P

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Warxcell, you're absolutely right! Handling these errors in the Init function would indeed be a more robust approach. It allows for better error management during initialization and avoids the use of panic in the Middleware function. Thanks for pointing that out!


Learnings added
Learnt from: Warxcell
PR: roadrunner-server/static#126
File: plugin.go:0-0
Timestamp: 2024-08-09T18:49:06.103Z
Learning: Errors related to creating the immutable server in the `Middleware` function should be handled in the `Init` function instead of using `panic`.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

plugin.go Outdated
server = createImmutableServer(s)
immutableServer, err := createImmutableServer(s)
if err != nil {
panic(err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Panic in Go is not the same as exception in PHP, for example, and should be avoided as much as possible. All data should be initialized (since you know the list of files to serve on the Init stage) on the Init stage. Middleware, on error should delete the buggy file from the list of the served files (or try to re-fetch).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. This way should be ok then?

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 3fc3c06 and 548cea3.

Files selected for processing (1)
  • plugin.go (6 hunks)
Files skipped from review as they are similar to previous changes (1)
  • plugin.go

@Warxcell Warxcell requested a review from rustatian August 9, 2024 18:56
plugin.go Outdated
etag string
}

func createImmutableServer(s *Plugin) (FileServer, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing plugin pointer is a bad idea. Consider approach without that.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like that?

@rustatian
Copy link
Member

Test should also be added. You may copy static files tests from the HTTP plugin and add new tests for an immutable option.

@Warxcell
Copy link
Author

Warxcell commented Aug 9, 2024

Test should also be added. You may copy static files tests from the HTTP plugin and add new tests for an immutable option.

I've searched for tests but I could not find any tests. Like, where existing test for the middleware lives?

@rustatian
Copy link
Member

There are plenty of tests here: https://github.com/roadrunner-server/http/tree/master/tests. They all live in the root plugin. Yeah, no need to copy them to this middleware, just add new to the http plugin. Use replace go.mod directive to run tests with the new functionality on your local machine.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 548cea3 and 6b3ecce.

Files selected for processing (1)
  • plugin.go (6 hunks)
Files skipped from review as they are similar to previous changes (1)
  • plugin.go

@Warxcell
Copy link
Author

Warxcell commented Aug 9, 2024

There are plenty of tests here: https://github.com/roadrunner-server/http/tree/master/tests. They all live in the root plugin. Yeah, no need to copy them to this middleware, just add new to the http plugin. Use replace go.mod directive to run tests with the new functionality on your local machine.

Ok, so my idea is that there shouldn't be any difference between immutable true/false (of course newly added files won't be served - but I will add just one case for it) - so I want to run all existing tests once with immutable: false and once with immutable: true

How would to override

http.static.immutable

in

	cfg := &config.Plugin{
		Version: "2023.3.5",
		Path:    "configs/.rr-http-static.yaml",
	}

	err := cont.RegisterAll(
		cfg,
		&logger.Plugin{},
		&server.Plugin{},
		&httpPlugin.Plugin{},
		&gzip.Plugin{},
		&static.Plugin{},
	)

Basically what's the programmatic way of using: -o http.static.immutable=true

https://docs.roadrunner.dev/docs/general/config#cli-command-and-arguments ?

@Warxcell Warxcell requested a review from rustatian August 9, 2024 20:07
@rustatian
Copy link
Member

Hey @Warxcell,
You may write overrides directly to the config plugin.

	cfg := &config.Plugin{
		Version: "2023.3.5",
		Path:    "configs/.rr-http-static.yaml",
		-- here
	}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: 🏗 In progress
Development

Successfully merging this pull request may close these issues.

[💡 FEATURE REQUEST]: Static HTTP middleware, immutable option.
2 participants