I did what I wanted to do.
Basically, there is this online multilingual phrase dictionary at https://abc.times.lv. But the free version only returns one word at a time. One day I had the idea of creating a clever script. It works like this:
1)A user on my site requests a word to translate, meaning they pass it to my script.
2)The script contacts abc.times.lv, feeds the word to its dictionary, and finds out how many pages of resulting words there are
3)The script downloads all those pages in order and displays them in one window, nicely and without ads
However, this mechanism, implemented in PHP (which I don't know at all, by the way), was ungodly slow. No wonder! To do all that, the script has to download between 10 and 150 pages, each 50 KB in size, and it does so sequentially rather than in parallel.
In the end, the page took a minute to load, so I gave up on the idea. The rough Latvian version, however, is still lying around right here. The word abols (apple), with 34 translation variants, takes 42–70 seconds to load.
Even then, I had the idea of loading pages in parallel, as modern download managers like FlashGet do by downloading different parts of a file simultaneously. But with PHP, the possibility of doing this ranges from "difficult" to "impossible" depending on the hosting environment.
So now that I've finally got around to evaluating the ASP.NET development environment, I decided to implement the idea.
The result is an ASP.NET program written in C#, running on my home computer.
For now, only the Latvian–Russian dictionary is supported.
The new algorithm works like this:
1)It downloads the first page and finds out how many pages there will be in total.
2)It creates as many threads as there are pages and makes each thread download its own page. As a result, everything is downloaded more or less simultaneously.
3)It displays the result.
Time taken: 25–30 seconds. Alas, for some reason the improvement is modest. Still, the comparison isn't entirely fair, because the two versions were run on completely different computers in different locations.
Out of curiosity, I made a version that works the old way. Its time became comparable to the PHP version, which makes sense, since everything comes down to download speed rather than program execution speed.
Basically, it isn't usable yet. The next logical step is to cache words. But that would already be the final stage of copyright infringement, so I wouldn't be able to release something like that publicly.
Good news, everyone! Big and dull programming article follows!
2006-03-16