Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 30 Dec 2018 17:26:01 -0800
From:      Mark Millard <marklmi@yahoo.com>
To:        mmel@freebsd.org
Cc:        freebsd-emulation@freebsd.org, FreeBSD Current <freebsd-current@freebsd.org>, ports-list freebsd <freebsd-ports@freebsd.org>, freebsd-arm <freebsd-arm@freebsd.org>, FreeBSD Toolchain <freebsd-toolchain@freebsd.org>
Subject:   Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved) [details of a specific qemu-arm-static source code problem]
Message-ID:  <A6CDEEBE-1753-4D28-9F9C-90E7789BB7D2@yahoo.com>
In-Reply-To: <CD78D55F-9B73-4914-9E7A-41590F058B03@yahoo.com>
References:  <FF9B4284-4E6B-4D36-86A0-18861B527AC0@yahoo.com> <865A13C8-9749-486E-9F79-5EEDDECBE621@yahoo.com> <0154C3AC-D85B-4FCF-BA63-454BC26BC1A2@yahoo.com> <A6A58CE3-062B-4B79-A8C2-ADFDAA04C6AF@yahoo.com> <13f5e4dd-33fb-2170-e31a-1b5d5f155869@freebsd.org> <ABA957EA-B8EE-4B8C-9C2F-B745BA652BF6@yahoo.com> <2E3F6196-4652-40D2-937F-8860B6005A35@yahoo.com> <CD78D55F-9B73-4914-9E7A-41590F058B03@yahoo.com>

next in thread | previous in thread | raw e-mail | index | archive | help
[Removing __packed did make the size and offsets match armv7
and the build worked based on the reconstructed qemu-arm-static.]

On 2018-Dec-30, at 16:38, Mark Millard <marklmi@yahoo.com> wrote:

> On 2018-Dec-28, at 12:12, Mark Millard <marklmi at yahoo.com> wrote:
>=20
>> On 2018-Dec-28, at 05:13, Michal Meloun <melounmichal at gmail.com> =
wrote:
>>=20
>>> Mark,
>>> this is known problem with qemu-user-static.
>>> Emulation of every single interruptible syscall is broken by design =
(it
>>> have signal related races). Theses races cannot be solved without =
major
>>> rewrite of syscall emulation code.
>>> Unfortunately, nobody actively works on this, I think.
>>>=20
>>=20
>> Thanks for the note setting some expectations.
>> . . .
>=20
>=20
> It turns out that I've been through (part of?) this before and
> mikael.urankar@gmail.com had back then provided a qemu-user-static
> patch (that might have been arm specific or 32-bit target specific
> when running on a 64-bit host). (The qemu-user-static code structure
> seems to have changed some afterwards and the patch is no longer
> where he had pointed me to back then.)
>=20
> To show size and offsets on armv7 vs. armd64 for struct kevent
> I use:
>=20
> # more kevent_size_offsets.c=20
> #include "/usr/include/sys/event.h" // kevent
> #include <stddef.h> // offsetof
> #include <stdio.h>  // printf
>=20
> int
> main()
> {
>        printf("%lu\n", (unsigned long) sizeof(struct kevent));
>        printf("ident %lu\n", (unsigned long) offsetof(struct kevent, =
ident));
>        printf("filter %lu\n", (unsigned long) offsetof(struct kevent, =
filter));
>        printf("flags %lu\n", (unsigned long) offsetof(struct kevent, =
flags));
>        printf("fflags %lu\n", (unsigned long) offsetof(struct kevent, =
fflags));
>        printf("data %lu\n", (unsigned long) offsetof(struct kevent, =
data));
>        printf("udata %lu\n", (unsigned long) offsetof(struct kevent, =
udata));
>        printf("ext %lu\n", (unsigned long) offsetof(struct kevent, =
ext));
>        return 0;
> }
>=20
> It ends up showing on armv7 (under qemu-arm-static insteead of native, =
not
> that it matters here):
>=20
> # ./a.out
> 64
> ident 0
> filter 4
> flags 6
> fflags 8
> data 16
> udata 24
> ext 32
>=20
> On amd64 (native) it ends up as:
>=20
> # ./a.out
> 64
> ident 0
> filter 8
> flags 10
> fflags 12
> data 16
> udata 24
> ext 32
>=20
> Thus a translation of layout is required when hosted. This is for:
>=20
> struct kevent {
>        __uintptr_t     ident;          /* identifier for this event */
>        short           filter;         /* filter for event */
>        unsigned short  flags;          /* action flags for kqueue */
>        unsigned int    fflags;         /* filter flag value */
>        __int64_t       data;           /* filter data value */
>        void            *udata;         /* opaque user data identifier =
*/
>        __uint64_t      ext[4];         /* extensions */
> };
>=20
> But qemu-user-static has for translation purposes:
>=20
> struct target_freebsd_kevent {
>    abi_ulong  ident;
>    int16_t    filter;
>    uint16_t   flags;
>    uint32_t   fflags;
>    int64_t data;
>    abi_ulong  udata;
>    uint64_t  ext[4];
> } __packed;
>=20
> (note the __packed) for which in amd64's qemu_arm_static has
> the size and offsets:
>=20
> # gdb qemu-arm-static
> . . .
> (gdb) p/d sizeof(struct target_freebsd_kevent)
> $1 =3D 56
> (gdb) p/d &((struct target_freebsd_kevent *)0)->ident
> $2 =3D 0
> (gdb) p/d &((struct target_freebsd_kevent *)0)->filter
> $3 =3D 4
> (gdb) p/d &((struct target_freebsd_kevent *)0)->flags
> $4 =3D 6
> (gdb) p/d &((struct target_freebsd_kevent *)0)->fflags
> $5 =3D 8
> (gdb) p/d &((struct target_freebsd_kevent *)0)->data
> $6 =3D 12
> (gdb) p/d &((struct target_freebsd_kevent *)0)->udata
> $7 =3D 20
> (gdb) p/d &((struct target_freebsd_kevent *)0)->ext
> $8 =3D 24
>=20
> which which does not match the armv7 offsets for
> data, udata, or ext and does not have the right size
> for struct target_freebsd_kevent[] indexing to
> match armv7's struct target_freebsd_kevent[] indexing.
>=20
> This in turn makes the do_freebsd_kevent code do the wrong
> thing in its:
>=20
>    struct target_freebsd_kevent *target_changelist, *target_eventlist;
> . . .
>        for (i =3D 0; i < arg3; i++) {
>            __get_user(changelist[i].ident, =
&target_changelist[i].ident);
>            __get_user(changelist[i].filter, =
&target_changelist[i].filter);
>            __get_user(changelist[i].flags, =
&target_changelist[i].flags);
>            __get_user(changelist[i].fflags, =
&target_changelist[i].fflags);
>            __get_user(changelist[i].data, &target_changelist[i].data);
>            /* __get_user(changelist[i].udata, =
&target_changelist[i].udata); */
> #if TARGET_ABI_BITS =3D=3D 32
>            changelist[i].udata =3D (void =
*)(uintptr_t)target_changelist[i].udata;
>            tswap32s((uint32_t *)&changelist[i].udata);
> #else
>            changelist[i].udata =3D (void =
*)(uintptr_t)target_changelist[i].udata;
>            tswap64s((uint64_t *)&changelist[i].udata);
> #endif
>            __get_user(changelist[i].ext[0], =
&target_changelist[i].ext[0]);
>            __get_user(changelist[i].ext[1], =
&target_changelist[i].ext[1]);
>            __get_user(changelist[i].ext[2], =
&target_changelist[i].ext[2]);
>            __get_user(changelist[i].ext[3], =
&target_changelist[i].ext[3]);
>        }
> . . .
>        for (i =3D 0; i < arg5; i++) {
>            __put_user(eventlist[i].ident, &target_eventlist[i].ident);
>            __put_user(eventlist[i].filter, =
&target_eventlist[i].filter);
>            __put_user(eventlist[i].flags, &target_eventlist[i].flags);
>            __put_user(eventlist[i].fflags, =
&target_eventlist[i].fflags);
>            __put_user(eventlist[i].data, &target_eventlist[i].data);
>            /* __put_user(eventlist[i].udata, =
&target_eventlist[i].udata);*/
> #if TARGET_ABI_BITS =3D=3D 32
>            tswap32s((uint32_t *)&eventlist[i].data);
>            target_eventlist[i].data =3D (uintptr_t)eventlist[i].data;
> #else
>            tswap64s((uint64_t *)&eventlist[i].data);
>            target_eventlist[i].data =3D (uintptr_t)eventlist[i].data;
> #endif
>            __put_user(eventlist[i].ext[0], =
&target_eventlist[i].ext[0]);
>            __put_user(eventlist[i].ext[1], =
&target_eventlist[i].ext[1]);
>            __put_user(eventlist[i].ext[2], =
&target_eventlist[i].ext[2]);
>            __put_user(eventlist[i].ext[3], =
&target_eventlist[i].ext[3]);
>        }
>=20
>=20
>=20
> I'll eventually do something to have struct target_freebsd_kevent for
> amd64-native targeting armv7 and see if that is sufficient to avoid =
the
> problem in my context. Previously removing the __packed was enough to
> make the structure the same size with the same offsets as for armv7.
> (Such might not have been appropriate to all targets.)
>=20
> armv6 would have the same problem as I understand things.


Using commented out __packed in:

struct target_freebsd11_kevent {
    abi_ulong  ident;
    int16_t    filter;
    uint16_t   flags;
    uint32_t   fflags;
    abi_long   data;
    abi_ulong  udata;
} ; // __packed;

struct target_freebsd_kevent {
    abi_ulong  ident;
    int16_t    filter;
    uint16_t   flags;
    uint32_t   fflags;
    int64_t data;
    abi_ulong  udata;
    uint64_t  ext[4];
} ; // __packed;


in =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-4ef7d07/b=
sd-user/syscall_defs.h
was sufficient to allow the multimedia/gstreamer1-qt@qt5 build to =
complete: no more hang-up.

So this is likely what is wrong for the packages-builders as well.


=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A6CDEEBE-1753-4D28-9F9C-90E7789BB7D2>