Date: Fri, 20 Feb 2009 18:01:53 -0500 From: Junsuk Shin <junsukshin@gmail.com> To: Pieter de Goeje <pieter@degoeje.nl>, freebsd-questions@freebsd.org Subject: Re: read() vs fread() Message-ID: <7873ac110902201501l74a79ffyfca138331386c317@mail.gmail.com> In-Reply-To: <200902202209.12765.pieter@degoeje.nl> References: <7873ac110902201207t61038469o6a94d81bcd392941@mail.gmail.com> <200902202209.12765.pieter@degoeje.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
setvbuf(file, buf, _IOFBF, bufsize) solved the problem perfectly.
Thanks a lot.
On Fri, Feb 20, 2009 at 4:09 PM, Pieter de Goeje <pieter@degoeje.nl> wrote:
>
> On Friday 20 February 2009 21:07:57 Junsuk Shin wrote:
> > Hi BSD guys,
> >
> > While I was doing simple file read test, I found that there is a huge
> > difference in file read performance between read() and fread(). I'm
> > wondering if I'm doing something wrong or if someone has experienced
> > similar things.
> >
> > Here is what I did,
> >
> > For the specific application, I need to bypass cache (I read only
> > once, and that's all)
> > The test file is 700Mbytes dummy file.
> > Test app just reads the whole file.
> >
> > Test is done on FreeBSD 7.1 amd 64, Celeron E1200, WD Caviar SE16 SATA 7200
> > RPM
> >
> > For test 1,
> >
> > fd = open(name, O_RDONLY | O_DIRECT);
> > while(...) {
> > cnt = read(....);
> > ....
> > }
> >
> > for test 2,
> >
> > fd = open(name, O_RDONLY | O_DIRECT);
> > file = fdopen(fd,"r");
> > while(...) {
> > cnt = fread(....);
> > ....
> > }
> >
> > test 1 takes about 11.64 seconds (63 MBytes/s), and test 2 takes about
> > 51.53 seconds (14 MBytes/s)
> >
> > If I use the pair of fopen() and fread(), it will have cache effect,
> > so the result doesn't say much of hdd performance.
> >
> > Personally, I don't think the overhead of fread() (wrapper in libc) is
> > that huge. What would be the reason for this?
>
> The reason is that by default a FILE has a really small internal buffer. Take
> a look at gstat(8) while running the test: you can clearly see an insane
> amount of I/O requests being done (almost 5000 reads per second on my HDD).
> To solve this call setvbuf(3):
>
> setvbuf(file, buf, _IOFBF, bufsize);
>
> A bufsize of 16k or bigger should help a lot. After this modification, I see
> about 900 reads per second (using bufsize = 64k) and the read speed is equal
> to the read(2) case.
>
> Regards,
>
> Pieter de Goeje
--
Junsuk
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7873ac110902201501l74a79ffyfca138331386c317>