Platform for energy data exchange
Background
Customer Request
Challenges & Solutions
Challenges
No processes for system setup
There were no processes for system setup in place, so users couldn’t set up or start the system by themselves and often had to ask the technical specialists for help, which required time & resources.
Absent file transfer process
The platform had no structured process for file transfers. Data security was compromised – data was duplicated, and it was hard to control access. It also caused confusion, risks with meeting compliance requirements, and required more time.
Undermined scalability by complex data exchange
There were no access controls in place. The users shared files with partners using two different methods — AWS S3 and SFTP – for different user requirements. Nevertheless, using both of them made the process less flexible.
Solutions
Custom Kubernetes Operator
We developed a specialized K8s operator that guides users through configuration in an automated way. It removes the need to navigate raw Git files and lets teams launch the service confidently, with minimal friction or outside support.
Scheduler system for parallel file transfer
For file transfer, we created a smart system that can move several files at once, prevents conflicts by making sure no two jobs interfere with each other, and can automatically resume jobs if something goes wrong.
Integrating SFTPGO for file sharing
We simplified the process using SFTPGO to enable users access S3 storage through SFTP protocol, too. It helped to connect both methods and made sharing files more flexible.
project facts
main features
Comprehensive permission system
We built an extensive security framework with customizable access levels, enabling both external and internal teams to safely access only the files they need. It ensures data security while reducing the need for complex manual setups.
Self-Service Setup
Once the service is up and running, teams can onboard and configure it without constant technical help. Settings are flexible but simple, so support teams aren’t overloaded, and new users can start quickly with minimal guidance.
Customized FileGateway Service for unified file access
We built a file sharing service (FileGateway) that works with both modern and traditional systems. External partners or internal users can access the same files using either AWS S3 or the older SFTP method, whichever works best for them. Behind the scenes, files are stored in one secure place, but users don’t need to worry about how it works.
Technologies used
