Re: FD_SETSIZE on Linux?

Hi,

I noticed when trying to set pgbench's client count to a high number, I had
to reduce it, and I found the maximum I can get away with is 1014. Any
higher and I get:

invalid number of clients: 1015

I find this in pgbench.c:

#ifdef FD_SETSIZE
#define MAXCLIENTS (FD_SETSIZE - 10)
#else
#define MAXCLIENTS 1024
#endif

And FS_SETSIZE defined before it:

#ifdef WIN32
#define FD_SETSIZE 1024 /* set before winsock2.h is
included */
#endif /* ! WIN32 */

... but apparently only if using Windows, which I'm not.

So it appears that MAXCLIENTS is being set to 1014 (1024 - 10), which looks
like should only be the case on Windows.

I'm a bit confused here. Shouldn't my MAXCLIENTS be set to 1024?

Thom

Thom Brown <thom(at)linux(dot)com> writes:
> I find this in pgbench.c:

> #ifdef FD_SETSIZE
> #define MAXCLIENTS (FD_SETSIZE - 10)
> #else
> #define MAXCLIENTS 1024
> #endif

FD_SETSIZE is supposed to be defined, according to the POSIX spec:

The <sys/select.h> header shall define the following symbolic constant,
which shall have a value suitable for use in #if preprocessing directives:

FD_SETSIZE
Maximum number of file descriptors in an fd_set structure.

It looks like Linux sets it to 1024. On RHEL6, at least, I find this:

$ grep -r FD_SETSIZE /usr/include
/usr/include/linux/posix_types.h:#undef __FD_SETSIZE
/usr/include/linux/posix_types.h:#define __FD_SETSIZE 1024
...
/usr/include/sys/select.h:#define FD_SETSIZE __FD_SETSIZE
...

> #ifdef WIN32
> #define FD_SETSIZE 1024 /* set before winsock2.h is included */
> #endif /* ! WIN32 */

Windows probably hasn't got sys/select.h at all, so it may not provide
this symbol.

Interestingly, it looks like POSIX also requires <sys/time.h> to define
FD_SETSIZE. I wonder whether Windows has that header? It'd definitely
be better to get this symbol from the system than assume 1024 will work.

regards, tom lane

On 10 September 2014 00:21, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Thom Brown <thom(at)linux(dot)com> writes:
> > I find this in pgbench.c:
>
> > #ifdef FD_SETSIZE
> > #define MAXCLIENTS (FD_SETSIZE - 10)
> > #else
> > #define MAXCLIENTS 1024
> > #endif
>
> FD_SETSIZE is supposed to be defined, according to the POSIX spec:
>
> The <sys/select.h> header shall define the following symbolic constant,
> which shall have a value suitable for use in #if preprocessing
> directives:
>
> FD_SETSIZE
> Maximum number of file descriptors in an fd_set structure.
>
> It looks like Linux sets it to 1024. On RHEL6, at least, I find this:
>
> $ grep -r FD_SETSIZE /usr/include
> /usr/include/linux/posix_types.h:#undef __FD_SETSIZE
> /usr/include/linux/posix_types.h:#define __FD_SETSIZE 1024
> ...
> /usr/include/sys/select.h:#define FD_SETSIZE
> __FD_SETSIZE
> ...
>

Ah yes, I have the same on Debian:

/usr/include/linux/posix_types.h:#undef __FD_SETSIZE
/usr/include/linux/posix_types.h:#define __FD_SETSIZE 1024
...
usr/include/x86_64-linux-gnu/sys/select.h:#define FD_SETSIZE __FD_SETSIZE
/usr/include/x86_64-linux-gnu/bits/typesizes.h:#define __FD_SETSIZE 1024
...

I didn't think to look beyond Postgres' code.

> > #ifdef WIN32
> > #define FD_SETSIZE 1024 /* set before winsock2.h is
> included */
> > #endif /* ! WIN32 */
>
> Windows probably hasn't got sys/select.h at all, so it may not provide
> this symbol.
>
> Interestingly, it looks like POSIX also requires <sys/time.h> to define
> FD_SETSIZE. I wonder whether Windows has that header? It'd definitely
> be better to get this symbol from the system than assume 1024 will work.
>

Okay, this now makes sense. It just takes the system value and reduces it
by 10 to get the MAXCLIENTS value.

Thanks for the explanation.

Thom