Expose internal modules?

[lukas2511]

Hi there,

I'm currently building a replacement for mirrorbits to use for the media.ccc.de cdn, and I was looking for a way to cleanly scan our rsync mirrors for available files. For that I really wanted to get a native data structure instead of relying on parsing human readable / formatted output as its also done in mirrorbits.

For now I've just added some code to a local fork of this project, using various internal modules, and it's been working fine. I'd love to keep this in a clean seperate public repository but since the modules are internal that doesn't seem possible.

Would you maybe be able to make the internal modules available for use in other projects?

Currently I'm using these modules:

• github.com/gokrazy/rsync/internal/progress
• github.com/gokrazy/rsync/internal/receiver
• github.com/gokrazy/rsync/internal/rsyncopts
• github.com/gokrazy/rsync/internal/rsyncos
• github.com/gokrazy/rsync/internal/rsyncwire

Thanks,
Lukas

[stapelberg]

Hey @lukas2511! Glad to hear the project is useful for media.ccc.de :)

I am generally in favor of providing useful functionality for embedding in other projects, but opening up the internal packages as-is would not be a good move: their API is not cleaned up yet, so by opening it up, we’d restrict the changes we can make to the project itself.

So I think we should figure out what a good API for your use-case looks like. Can you show me how you’re using the code, and what an ideal API would look like from your perspective please?

[lukas2511]

I've written a function query_mirror_files that takes in:

• Host
• Port
• Path
• Username
• Password

And that returns a map of paths to *receiver.File objects, which contain all the information I need (and more).

I've also implemented MD4 auth in that codebase (as that seems to be the only method available when announcing as rsync version 27).

It's still quite ugly, I've not written much Go code yet, so I'm also not creating a project for now, but I've put the code into a gist: https://gist.github.com/lukas2511/5d1d669112e22e328c72e46afbf15a1c