3

I'm trying to build a Java application that connects to a remote database that stores and retrieves video files that can be over 200MB. I could store/retrieve these files in MySQL directly as LONGBLOBs, yet I read again and again how that's bad practice for various reasons. One solution commonly offered is to store the files on the filesystem and have the database store the location file and serve it up from the server directly.

My problem is, I'm very new to storage and am completely unsure of how to go about doing that. Should I make an FTP connection to the front end once a video file has been requested, then deliver that video over that connection? Or are people usually talking about something else when they say to pull it from the filesystem?

I'm using Java and JDBC to make calls to MySQL.

Doc Brown
  • 199,015
  • 33
  • 367
  • 565
dyslexit
  • 139
  • 1
  • 3
  • You've told us about the technology you are intending to use, but not much about the expected behaviour. Who, What and Why. Do you need to stream the videos. Do you need content control. Encryption, Transcoding. – Martin Spamer Apr 06 '17 at 19:46
  • 5
    This question is asked almost every week on Stack Overflow in some form (should I store large files in the database or on the file system?). The answer is almost always the same: store large files on the file system, and store a link to the file system file in the database. The reasons are self-evident: the file system is designed to handle... files. – Robert Harvey Apr 06 '17 at 21:20
  • @RobertHarvey: I took the freedom to change the title to match the actual question, seems it was a slightly different one. But maybe it would be better suited for SO. – Doc Brown Apr 07 '17 at 06:33
  • @DocBrown You're right, I was asking _how_ to store videos, not _which way_ to do it. I probably should've asked on SO, but MartinSpamer made me realize I need to flesh much more out about my project first before I ask again. – dyslexit Apr 07 '17 at 18:39

1 Answers1

4

No, store the files directly on a filesystem and save relative paths to them. This has several benefits:

  • faster database migration as it does not contain TBs of data,
  • not being directly limited by the location of the storage space - you simply prepend the root to the relative path and work with that one (serve that one to the application clients),
  • for certain resources, native HTTP caching thanks to Last-Modified/ETag and If-Modified-Since/If-None-Match headers, thus saving client resources1,
  • in your case easier implementation of content streaming - by allowing clients of the application to retrieve sections of the video files, by cutting the video file and returning only specific byte range of it by the server.

With files being stored on the filesystem you need to make sure they are in a directory which is publicly available (should it be, of course). You then serve these files directly from the filesystem from their path.

Let's say your application is located at /home/dyslexit/www, where www is a publicly available directory from the browser. Within the www directory you would create a new one, eg. uploads, which would also be publicly available and used to store the files. Should you then serve some file to a client of the application it would be as easy as providing them a URI: https://domain.com/uploads/videos/my-video.mp4, which would translate to the actual /home/dyslexit/www/uploads/videos/my-video.mp4 file on the server.

It is however not required to directly copy the structure. Should you need it instead of directly serving the files from a HDD when a file is requested you could instead forward the request to a script checking permissions of the requesting user, denying them access should they not be authorized to access a resource or forwarding them to the file being located anywhere respectively.

1 Obviously this can also be implemented if you store files in a database but requires additional setup and custom header manipulation. When serving files directly from the file system this is mostly native.

Either way, if your business goal is not actually storing files - such as your are creating a data-storage system - but it is merely a side-effect of the business (let's say you want videos in a Facebook-like application), the better choice would probably be to completely outsource file storage to a 3rd party system dedicated to this feature - such as Amazon S3.

Andy
  • 10,238
  • 4
  • 25
  • 50
  • 2
    I don't see how this answers the question (which was **not** *"should I store large files in the database or on the file system?"*). To my understanding, the question was how to transfer files from the server to the client when the client has only a JDBC connection; for example, is there is an extra protocol like ftp required? Or do all clients need an SMB or NFS share? Will one need a web server for making files available through http? Etc. – Doc Brown Apr 07 '17 at 22:36
  • @DocBrown I have added some more points to the answer, however as I understand it the user is actually building the application which is serving the files and is asking whether they should store the files in the database or on a filesystem. – Andy Apr 08 '17 at 06:37