robotwera.blogg.se

Each separate 'stream' (or really, file) in the multistream dump contains 100 pages, except possibly the last one.įor multistream, you can get an index file, 2. NOTE THAT the multistream dump file contains multiple bz2 'streams' (bz2 header, body, footer) concatenated together into one file, in contrast to the vanilla file which contains one stream. And it will unpack to ~5-10 times its original size. You might be tempted to get the smaller non-multistream archive, but this will be useless if you don't unpack it. The only downside to multistream is that it is marginally larger. Your reader should handle this for you, if your reader doesn't support it it will work anyway since multistream and non-multistream contain the same xml. But with multistream, it is possible to get an article from the archive without unpacking the whole thing. So if you unpack either, you get the same data. 2 and 2 both contain the same xml contents. GET THE MULTISTREAM VERSION! (and the corresponding index file, 2)