10

We currently abuse our version control system and use it... pretty much as an FTP to store big release binaries (4+ GB).

We are seeking to move away from this horrendous practice which has and continues to devolve and take more and more of IT's resources while crippling the integration and release processes.

A solution for this would be using P2P file sharing to distribute these release images/binaries, mixing file servers and some key user machines as seeds.

So my question comes in two parts:

  1. Have any of you resorted to setting up BitTorrent in your intranet to distribute release images/binaries?, if not, what do you think about this idea (which was not actually mine BTW, but I think it is great)
  2. Is there open source web-based software for handling publishing of BitTorrent trackers?, so that when there is a new release it can be searchable and display its availability with regards to seeds and leeches? (...you know what I'm referring to)

Edit: The intranet is global (e.g. USA, China, Germany, Mexico). Regular FTP could work but it is not as cost-effective.

dukeofgaming
  • 13,943
  • 6
  • 50
  • 77

2 Answers2

9

I like the idea. BitTorrent is possibly the best solution for sharing large files among numerous users devised so far, it's a shame that the stigma it carries will probably continue hampering it's mainstream adoption. Still, many open source projects, including Linux distributions, use torrents as a secondary channel of distribution

You can find several open source trackers on sourceforge, among them TBSource and TBDev.net, current incarnations of popular torrentbits project. Another option is XBTT, although I suspect it might be an overkill for your needs. It has a spartan frontend, but there are third-party frontends available should you need them.

There's also a possibility to take advantage of torrents without having to setup and run a tracker, since torrent clients can operate in a trackerless mode, using a distributed hashtable to exchange peer information. You would still need to share torrent files somehow, but you can setup some simple website/ftp for that in no time, or store them on svn (not an issue since they weigh several kb per file).

scrwtp
  • 4,532
  • 1
  • 24
  • 29
1

Necro a very old question with an answer that privately predated the accepted answer by several years until it was publicly released about the time this question was asked and answered.

Twitter did this for some time for server deployments, reducing deployment time from 900 seconds to 12 seconds. I can't recall why they stopped doing it.

https://rubygems.org/gems/murder

https://github.com/lg/murder

https://vimeo.com/11280885

https://blog.twitter.com/engineering/en_us/a/2010/murder-fast-datacenter-code-deploys-using-bittorrent

I think there's also a clear need to publish and distribute large software installation artifacts to the global world using a bittorrent-based mediation (highly-available tracker) where random peers take the place of traditional CDNs.

EDIT: RIP Twitter

dhchdhd
  • 111
  • 3