Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 31 Aug 2017 16:37:28 -0700
From:      Mark Millard <markmi@dsl-only.net>
To:        Sean Bruno <sbruno@FreeBSD.org>
Cc:        Mark Linimon <linimon@lonesome.com>, Don Lewis <truckman@FreeBSD.org>, FreeBSD Toolchain <freebsd-toolchain@freebsd.org>, FreeBSD Ports <freebsd-ports@freebsd.org>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org>
Subject:   Re: FYI: qemu-ppc64-static and qemu-ppc-static "live-hang" when I attempt use with poudriere; qemu-arm-static and qemu-aarch64-static work
Message-ID:  <32379D16-A06E-4BE8-8FC5-C68A8B80E1D2@dsl-only.net>
In-Reply-To: <A61F781F-57C2-4611-8E44-C12E756E1569@dsl-only.net>
References:  <201708302332.v7UNWSVY073465@gw.catspoiler.org> <9B916738-394B-48B7-AA2E-6193F54760B3@dsl-only.net> <85B5ADE0-5573-4E04-8EC3-CB5751C035FF@dsl-only.net> <A61F781F-57C2-4611-8E44-C12E756E1569@dsl-only.net>

next in thread | previous in thread | raw e-mail | index | archive | help
[I show some of the target/ppc/translate.c source code
and related material this time. Not that I know enough
to patch it correctly.]

On 2017-Aug-31, at 12:13 PM, Mark Millard <markmi at dsl-only.net> =
wrote:

> [Turns out that the emulated program counter is not progressing
> for syscall emulation, at least for [some] syscall [failure] cases.]
>=20
> On 2017-Aug-30, at 8:43 PM, Mark Millard <markmi at dsl-only.net> =
wrote:
>=20
>> On 2017-Aug-30, at 4:32 PM, Don Lewis <truckman at FreeBSD.org> =
wrote:
>>=20
>>> On 30 Aug, Mark Millard wrote:
>>>> On 2017-Aug-30, at 4:00 AM, Mark Linimon <linimon at lonesome.com> =
wrote:
>>>>=20
>>>>> On Wed, Aug 30, 2017 at 03:09:40AM -0700, Mark Millard wrote:
>>>>>> It appears that qemu-ppc64-static and qemu-ppc-static from
>>>>>> emulators/qemu-user-static are broken.
>>>>>=20
>>>>> Correct, and known for some time.  (fwiw sparc64 hangs as well.)
>>>>=20
>>>> Looks like qemu-ppc64-static is stuck in a loop, calling
>>>> repeatedly:
>>>>=20
>>>> do_freebsd_syscall (cpu_env=3D0x860ea3ac0, num=3D58, arg1=3D14, =
arg2=3D35995509911, arg3=3D1024, arg4=3D268435904, arg5=3D281494784, =
arg6=3D35985701568, arg7=3D515, arg8=3D35985668288)
>>>> at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/syscall.c:210
>>>> 210	=
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/syscall.c: No such file or directory.
>>>>=20
>>>> Which is for:
>>>>=20
>>>> 58      AUE_READLINK    STD     { ssize_t readlink(char *path, char =
*buf, \
>>>>                                 size_t count); }
>>>>=20
>>>> As confirmed by (note the "callq  0x60207360 <readlink>" ):
>>>>=20
>>>> (gdb)=20
>>>> lock_user_string (guest_addr=3D14) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/qemu.h:508
>>>> 508	=
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/qemu.h: No such file or directory.
>>>>=20
>>>> (gdb) x/64i 0x0000000060045d3e
>>>> =3D> 0x60045d3e <do_freebsd_syscall+3246>:	callq  0x6004fd20 =
<target_strlen>
>>>> 0x60045d43 <do_freebsd_syscall+3251>:	test   %rax,%rax
>>>> 0x60045d46 <do_freebsd_syscall+3254>:	js     0x6004b99c =
<do_freebsd_syscall+26892>
>>>> 0x60045d4c <do_freebsd_syscall+3260>:	inc    %rax
>>>> 0x60045d4f <do_freebsd_syscall+3263>:	mov    $0x1,%edx
>>>> 0x60045d54 <do_freebsd_syscall+3268>:	mov    %rbx,%rdi
>>>> 0x60045d57 <do_freebsd_syscall+3271>:	mov    %rax,%rsi
>>>> 0x60045d5a <do_freebsd_syscall+3274>:	callq  0x6003c430 =
<page_check_range>
>>>> 0x60045d5f <do_freebsd_syscall+3279>:	test   %eax,%eax
>>>> 0x60045d61 <do_freebsd_syscall+3281>:	jne    0x6004bce4 =
<do_freebsd_syscall+27732>
>>>> 0x60045d67 <do_freebsd_syscall+3287>:	add    =
0x26d91b2(%rip),%rbx        # 0x6271ef20 <guest_base>
>>>> 0x60045d6e <do_freebsd_syscall+3294>:	je     0x6004bce4 =
<do_freebsd_syscall+27732>
>>>> 0x60045d74 <do_freebsd_syscall+3300>:	mov    $0x3,%edx
>>>> 0x60045d79 <do_freebsd_syscall+3305>:	mov    -0x2a8(%rbp),%r14
>>>> 0x60045d80 <do_freebsd_syscall+3312>:	mov    %r14,%rdi
>>>> 0x60045d83 <do_freebsd_syscall+3315>:	mov    %r12,%rsi
>>>> 0x60045d86 <do_freebsd_syscall+3318>:	callq  0x6003c430 =
<page_check_range>
>>>> 0x60045d8b <do_freebsd_syscall+3323>:	test   %eax,%eax
>>>> 0x60045d8d <do_freebsd_syscall+3325>:	jne    0x6004bce4 =
<do_freebsd_syscall+27732>
>>>> 0x60045d93 <do_freebsd_syscall+3331>:	add    =
0x26d9186(%rip),%r14        # 0x6271ef20 <guest_base>
>>>> 0x60045d9a <do_freebsd_syscall+3338>:	mov    =
-0x294(%rbp),%r10d
>>>> 0x60045da1 <do_freebsd_syscall+3345>:	mov    =
$0xfffffffffffffff2,%r13
>>>> 0x60045da8 <do_freebsd_syscall+3352>:	je     0x6004bcf2 =
<do_freebsd_syscall+27746>
>>>> 0x60045dae <do_freebsd_syscall+3358>:	mov    $0x602b93da,%esi
>>>> 0x60045db3 <do_freebsd_syscall+3363>:	mov    %rbx,%rdi
>>>> 0x60045db6 <do_freebsd_syscall+3366>:	callq  0x60230af0 =
<strcmp>
>>>> 0x60045dbb <do_freebsd_syscall+3371>:	test   %eax,%eax
>>>> 0x60045dbd <do_freebsd_syscall+3373>:	je     0x6004c566 =
<do_freebsd_syscall+29910>
>>>> 0x60045dc3 <do_freebsd_syscall+3379>:	mov    %rbx,%rdi
>>>> 0x60045dc6 <do_freebsd_syscall+3382>:	callq  0x60158660 <path>
>>>> 0x60045dcb <do_freebsd_syscall+3387>:	mov    %rax,%rdi
>>>> 0x60045dce <do_freebsd_syscall+3390>:	mov    %r14,%rsi
>>>> 0x60045dd1 <do_freebsd_syscall+3393>:	mov    %r12,%rdx
>>>> 0x60045dd4 <do_freebsd_syscall+3396>:	callq  0x60207360 =
<readlink>
>>>>=20
>>>> But note that the "lock_user_string (guest_addr=3D14)" and
>>>> "do_freebsd_syscall (cpu_env=3D0x860ea3ac0, num=3D58, arg1=3D14,"
>>>> indicate that the "readlink(char *path," is using a really
>>>> small address for the path string.
>>>>=20
>>>>=20
>>>> I've not figured a way for poudriere bulk builds to leave
>>>> behind the source code automatically. So far I've not
>>>> looked at the qemu-bsd-user source code. I do build with
>>>> both debug and optimization turned on via bsd.port.mk
>>>> having:
>>>=20
>>> The -w option will create a tarball of the work directory if the
>>> package build fails.  I also often use the testport -i option I want =
to
>>> poke around in the WRKDIR after a build.
>>=20
>> I've been using -w right along. But I'd not used testport at all.
>>=20
>> It looks to me like the syscall errno handling is messed
>> up. The details that I've observed follow. It follows
>> a simplified sequence of discovery as far a presentation
>> order goes.
>>=20
>> The looping code is:
>>=20
>> static inline void target_cpu_loop(CPUPPCState *env)
>> {
>>  CPUState *cs =3D CPU(ppc_env_get_cpu(env));
>>  target_siginfo_t info;
>>  int trapnr;
>>  target_ulong ret;
>>=20
>>  for(;;) {
>>      cpu_exec_start(cs);
>>      trapnr =3D cpu_exec(cs);
>>      cpu_exec_end(cs);
>>      process_queued_cpu_work(cs);
>>=20
>>      switch(trapnr) {
>> . . .
>>      case POWERPC_EXCP_SYSCALL_USER:
>>          /* system call in user-mode emulation */
>>          /* WARNING:
>>           * PPC ABI uses overflow flag in cr0 to signal an error
>>           * in syscalls.
>>           */
>>          env->crf[0] &=3D ~0x1;
>>          ret =3D do_freebsd_syscall(env, env->gpr[0], env->gpr[3], =
env->gpr[4],
>>                           env->gpr[5], env->gpr[6], env->gpr[7],
>>                           env->gpr[8], env->gpr[9], env->gpr[10]);
>>          if (ret =3D=3D (target_ulong)(-TARGET_QEMU_ESIGRETURN)) {
>>              /* Returning from a successful sigreturn syscall.
>>                 Avoid corrupting register state.  */
>>              break;
>>          }
>>          if (ret > (target_ulong)(-515)) {
>>              env->crf[0] |=3D 0x1;
>>              ret =3D -ret;
>>          }
>>          env->gpr[3] =3D ret;
>>          break;
>> . . .
>>      }
>>      process_pending_signals(env);
>>  }
>> }
>>=20
>> The observed env->gpr[3] =3D=3D 14 is from a prior loop
>> iteration having ret =3D=3D 14 in the:
>>=20
>>          env->gpr[3] =3D ret;
>>=20
>> Prior to this were the values (as seen via
>> lock_user_string):
>>=20
>> guest_addr=3D278408977
>> guest_addr=3D2
>>=20
>> That 2 also came from the prior ret =3D=3D 2 in the:
>>=20
>>          env->gpr[3] =3D ret;
>>=20
>> from when the 278408977 was in being attempted.
>>=20
>> For both the ret =3D=3D 2 and ret =3D=3D 14 were from:
>>=20
>>              ret =3D -ret;
>>=20
>> so the return values from do_freebsd_syscall were
>> -2 and -14 (interpreted as signed).
>>=20
>> The return values trace back to the following code,
>> where TARGET_EFAULT =3D=3D 14 :
>>=20
>> static inline abi_long do_bsd_readlink(CPUArchState *env, abi_long =
arg1,
>>      abi_long arg2, abi_long arg3)
>> {
>>  abi_long ret;
>>  void *p1, *p2;
>>=20
>>  LOCK_PATH(p1, arg1);
>>  p2 =3D lock_user(VERIFY_WRITE, arg2, arg3, 0);
>>  if (p2 =3D=3D NULL) {
>>      UNLOCK_PATH(p1, arg1);
>>      return -TARGET_EFAULT;
>>  }
>> #ifdef __FreeBSD__
>>  if (strcmp(p1, "/proc/curproc/file") =3D=3D 0) {
>>      CPUState *cpu =3D ENV_GET_CPU(env);
>>      TaskState *ts =3D (TaskState *)cpu->opaque;
>>      strncpy(p2, ts->bprm->fullpath, arg3);
>>      ret =3D MIN((abi_long)strlen(ts->bprm->fullpath), arg3);
>>  } else
>> #endif
>>  ret =3D get_errno(readlink(path(p1), p2, arg3));
>>  unlock_user(p2, arg2, ret);
>>  UNLOCK_PATH(p1, arg1);
>>=20
>>  return ret;
>> }
>>=20
>> The 2 is from:
>>=20
>>  ret =3D get_errno(readlink(path(p1), p2, arg3));
>>=20
>> At the time the p1 points to "/etc/malloc.conf":
>>=20
>> (gdb) step=20
>> path (name=3D0x10982f11 "/etc/malloc.conf") at util/path.c:173
>>=20
>> 169	const char *path(const char *name)
>> 170	{
>> 171	    /* Only do absolute paths: quick and dirty, but should =
mostly be OK.
>> 172	       Could do relative by tracking cwd. */
>> (gdb)=20
>> 173	    if (!base || !name || name[0] !=3D '/')
>> 174	        return name;
>> 175=09
>> 176	    return follow_path(base, name) ?: name;
>> 177	}
>>=20
>> (gdb) print base
>> $8 =3D (struct pathelem *) 0x0
>>=20
>> So name is returned unchanged.
>>=20
>>=20
>> The 2 is in turn from:
>>=20
>> #define	__ENOENT	2		/* No such file or =
directory */
>>=20
>>=20
>> Overall one oddity is that this code structure
>> seems to use -ret from:
>>=20
>>          ret =3D do_freebsd_syscall(env, env->gpr[0], env->gpr[3], =
env->gpr[4],
>>                           env->gpr[5], env->gpr[6], env->gpr[7],
>>                           env->gpr[8], env->gpr[9], env->gpr[10]);
>>=20
>> to retry the same operation again the next iteration,
>> but with env->gpr[3] =3D=3D -ret (as ret was on the return
>> of do_freebsd_syscall ).
>>=20
>> Once abs(ret) =3D=3D 14 it is fully stuck repeating itself.
>>=20
>> I've no clue if:
>>=20
>>          env->gpr[3] =3D ret;
>>=20
>> even makes sense here.
>>=20
>> I've not tried to track down the memory leak activity
>> that is associated.
>>=20
>> Nor have I checked anything for the:
>>=20
>>      cpu_exec_start(cs);
>>      trapnr =3D cpu_exec(cs);
>>      cpu_exec_end(cs);
>>      process_queued_cpu_work(cs);
>>=20
>> activity. It likely contributes to why the loop
>> retries the readlink again (with a junk address
>> for the path).
>=20
> I do not see activity advancing the emulated
> program counter as this looping/retrying happens.
> Nor anything that is adjusting the problematical
> re-used env->gpr[3] other than the:
>=20
> 516	            env->gpr[3] =3D ret;
>=20
> after the negation of ret for the syscall failure
> handling.
>=20
> This is confirmed by the following:
>=20
> (gdb) bt
> #0  cpu_tb_exec (cpu=3D0x860e9b8c0, itb=3D0x60723340 =
<static_code_gen_buffer+17488>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:141
> #1  0x0000000060039cb5 in cpu_loop_exec_tb (cpu=3D<optimized out>, =
tb=3D<optimized out>, last_tb=3D<optimized out>, tb_exit=3D<optimized =
out>)
>   at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:574
> #2  cpu_exec (cpu=3D<optimized out>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:672
> #3  0x000000006003c988 in target_cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/ppc/target_arch_cpu.h:139
> #4  cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/main.c:121
> #5  0x000000006003e003 in main (argc=3D<optimized out>, =
argv=3D<optimized out>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/main.c:516
> . . .
> (gdb) list
> 569	{
> 570	    uintptr_t ret;
> 571	    int32_t insns_left;
> 572=09
> 573	    trace_exec_tb(tb, tb->pc);
> 574	    ret =3D cpu_tb_exec(cpu, tb);
> 575	    tb =3D (TranslationBlock *)(ret & ~TB_EXIT_MASK);
> 576	    *tb_exit =3D ret & TB_EXIT_MASK;
> 577	    if (*tb_exit !=3D TB_EXIT_REQUESTED) {
> 578	        *last_tb =3D tb;
> . . .
> cpu_tb_exec (cpu=3D0x860e9b8c0, itb=3D0x60723340 =
<static_code_gen_buffer+17488>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:141
> 141	    CPUArchState *env =3D cpu->env_ptr;
> (gdb) print/x itb->pc
> $16 =3D 0x1074d784
> (gdb) c
> Continuing.
>=20
> Thread 1 hit Breakpoint 9, cpu_tb_exec (cpu=3D0x860e9b8c0, =
itb=3D0x60723340 <static_code_gen_buffer+17488>)
>   at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:141
> 141	    CPUArchState *env =3D cpu->env_ptr;
> . . .
> (gdb) print/x itb->pc
> $18 =3D 0x1074d784
> (gdb) c
> Continuing.
>=20
> Thread 1 hit Breakpoint 9, cpu_tb_exec (cpu=3D0x860e9b8c0, =
itb=3D0x60723340 <static_code_gen_buffer+17488>)
>   at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:141
> 141	    CPUArchState *env =3D cpu->env_ptr;
> (gdb) print/x itb->pc
> $19 =3D 0x1074d784
> (gdb) c
> Continuing.
>=20
> Thread 1 hit Breakpoint 9, cpu_tb_exec (cpu=3D0x860e9b8c0, =
itb=3D0x60723340 <static_code_gen_buffer+17488>)
>   at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:141
> 141	    CPUArchState *env =3D cpu->env_ptr;
> (gdb) print/x itb->pc
> $20 =3D 0x1074d784
>=20
> and so on.
>=20
> So it appears that syscall emulation does not progress the
> emulated instruction pointer and so the syscall repeats
> over and over.
>=20
> (I've still not tracked down what is leaking memory
> during this looping. But that is probably a secodnary
> concern at this point.)
>=20
>=20
> So how does the code get from:
>=20
> 139	        trapnr =3D cpu_exec(cs);
>=20
> to (re-)trying the failed syscall (readlink) attempt?
>=20
> (gdb) bt
> #0  0x00000000601e25c0 in siglongjmp ()
> #1  0x000000006003a1aa in cpu_loop_exit_restore (cpu=3D<optimized =
out>, pc=3D<optimized out>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec-common.c:77
> #2  0x00000000600e0eeb in raise_exception_err_ra (env=3D<optimized =
out>, exception=3D<optimized out>, error_code=3D0, raddr=3D0)
>   at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/excp_helper.c:905
> #3  helper_raise_exception_err (env=3D<optimized out>, =
exception=3D<optimized out>, error_code=3D0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/excp_helper.c:928
> #4  0x00000000607233e6 in static_code_gen_buffer ()
> #5  0x0000000060039ffa in cpu_tb_exec (cpu=3D0x860e9b8c0, =
itb=3D0x60723340 <static_code_gen_buffer+17488>)
>   at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:166
> #6  0x0000000060039cb5 in cpu_loop_exec_tb (cpu=3D<optimized out>, =
tb=3D<optimized out>, last_tb=3D<optimized out>, tb_exit=3D<optimized =
out>)
>   at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:574
> #7  cpu_exec (cpu=3D<optimized out>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:672
> #8  0x000000006003c988 in target_cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/ppc/target_arch_cpu.h:139
> #9  cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/main.c:121
> #10 0x000000006003e003 in main (argc=3D<optimized out>, =
argv=3D<optimized out>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/main.c:516
>=20
> It does a siglongjmp via helper_raise_execption_err :
>=20
> (gdb) up
> #1  0x0000000060039ffa in cpu_tb_exec (cpu=3D0x860e9b8c0, =
itb=3D0x60723340 <static_code_gen_buffer+17488>)
>   at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:166
> 166	    ret =3D tcg_qemu_tb_exec(env, tb_ptr);
> (gdb) list
> 161	        qemu_log_unlock();
> 162	    }
> 163	#endif /* DEBUG_DISAS */
> 164=09
> 165	    cpu->can_do_io =3D !use_icount;
> 166	    ret =3D tcg_qemu_tb_exec(env, tb_ptr);
> 167	    cpu->can_do_io =3D 1;
> 168	    last_tb =3D (TranslationBlock *)(ret & ~TB_EXIT_MASK);
> 169	    tb_exit =3D ret & TB_EXIT_MASK;
> 170	    trace_exec_tb_exit(last_tb, tb_exit);
> (gdb) print tb_ptr
> $11 =3D (uint8_t *) 0x607233c0 <static_code_gen_buffer+17616> =
"A\213n\354\205\355\017\214\037"
>=20
>  0x607233c0 <static_code_gen_buffer+17616>:	mov    -0x14(%r14),%ebp
>  0x607233c4 <static_code_gen_buffer+17620>:	test   %ebp,%ebp
>  0x607233c6 <static_code_gen_buffer+17622>:	jl     0x607233eb =
<static_code_gen_buffer+17659>
>  0x607233cc <static_code_gen_buffer+17628>:	movq   =
$0x1074d784,0x3c8(%r14)
>  0x607233d7 <static_code_gen_buffer+17639>:	mov    %r14,%rdi
>  0x607233da <static_code_gen_buffer+17642>:	mov    $0x203,%esi
>  0x607233df <static_code_gen_buffer+17647>:	xor    %edx,%edx
>  0x607233e1 <static_code_gen_buffer+17649>:	callq  0x600e0ed0 =
<helper_raise_exception_err>
> =3D> 0x607233e6 <static_code_gen_buffer+17654>:	jmpq   =
0x6071ef06 <static_code_gen_buffer+22>
>  0x607233eb <static_code_gen_buffer+17659>:	mov    $0x60723343,%eax
>  0x607233f0 <static_code_gen_buffer+17664>:	jmpq   0x6071ef08 =
<static_code_gen_buffer+24>
>=20
> The exception is exception=3D=3D515 . 515 is the
> figure matching up with POWERPC_EXCP_SYSCALL_USER .
>=20
> (gdb) stepi
> helper_raise_exception_err (env=3D0x860ea3ac0, exception=3D515, =
error_code=3D0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/excp_helper.c:927
> 927	{
> (gdb) bt
> #0  helper_raise_exception_err (env=3D0x860ea3ac0, exception=3D515, =
error_code=3D0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/excp_helper.c:927
> #1  0x00000000607233e6 in static_code_gen_buffer ()
> #2  0x0000000060039ffa in cpu_tb_exec (cpu=3D0x860e9b8c0, =
itb=3D0x60723340 <static_code_gen_buffer+17488>)
>   at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:166
> #3  0x0000000060039cb5 in cpu_loop_exec_tb (cpu=3D<optimized out>, =
tb=3D<optimized out>, last_tb=3D<optimized out>, tb_exit=3D<optimized =
out>)
>   at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:574
> #4  cpu_exec (cpu=3D<optimized out>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:672
> #5  0x000000006003c988 in target_cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/ppc/target_arch_cpu.h:139
> #6  cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/main.c:121
> #7  0x000000006003e003 in main (argc=3D<optimized out>, =
argv=3D<optimized out>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/main.c:516
>=20
> Later there is:
>=20
> raise_exception_err_ra (env=3D0x860ea3ac0, exception=3D515, =
error_code=3D0, raddr=3D0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/excp_helper.c:903
> 903	    cs->exception_index =3D exception;
>=20
> and then:
>=20
> (gdb) s
> cpu_loop_exit_restore (cpu=3D0x860e9b8c0, pc=3D0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec-common.c:74
> 74	    if (pc) {
> (gdb) n
> 77	    siglongjmp(cpu->jmp_env, 1);
> (gdb) n
> 0x00000000600398e9 in cpu_exec (cpu=3D0x860e9b8c0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:645
> 645	    if (sigsetjmp(cpu->jmp_env, 0) !=3D 0) {
> (gdb) bt
> #0  0x00000000600398e9 in cpu_exec (cpu=3D0x860e9b8c0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:645
> #1  0x000000006003c988 in target_cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/ppc/target_arch_cpu.h:139
> #2  cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/main.c:121
> #3  0x000000006003e003 in main (argc=3D<optimized out>, =
argv=3D<optimized out>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/main.c:516
> (gdb) n
> 651	        cpu =3D current_cpu;
> (gdb)=20
> 652	        cc =3D CPU_GET_CLASS(cpu);
> (gdb)=20
> 658	        cpu->can_do_io =3D 1;
> (gdb)=20
> 659	        tb_lock_reset();
> (gdb)=20
> 660	        if (qemu_mutex_iothread_locked()) {
> (gdb)=20
> 661	            qemu_mutex_unlock_iothread();
> (gdb)=20
> 666	    while (!cpu_handle_exception(cpu, &ret)) {
> (gdb)=20
> 679	    cc->cpu_exec_exit(cpu);
> (gdb) n
> 680	    rcu_read_unlock();
> (gdb) n
> 683	}
> (gdb) n
> target_cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/ppc/target_arch_cpu.h:140
> 140	        cpu_exec_end(cs);
>=20
> And it sends up back in:
>=20
> 141		process_queued_cpu_work(cs);
> 142=09
> 143	        switch(trapnr) {
> . . .
> 497	        case POWERPC_EXCP_SYSCALL_USER:
> 498	            /* system call in user-mode emulation */
> 499	            /* WARNING:
> 500	             * PPC ABI uses overflow flag in cr0 to signal an =
error
> 501	             * in syscalls.
> 502	             */
> (gdb)=20
> 503	            env->crf[0] &=3D ~0x1;
> 504	            ret =3D do_freebsd_syscall(env, env->gpr[0], =
env->gpr[3], env->gpr[4],
> 505	                             env->gpr[5], env->gpr[6], =
env->gpr[7],
> 506	                             env->gpr[8], env->gpr[9], =
env->gpr[10]);
> 507	            if (ret =3D=3D =
(target_ulong)(-TARGET_QEMU_ESIGRETURN)) {
> 508	                /* Returning from a successful sigreturn =
syscall.
> 509	                   Avoid corrupting register state.  */
> 510	                break;
> 511	            }
> 512	            if (ret > (target_ulong)(-515)) {
> (gdb)=20
> 513	                env->crf[0] |=3D 0x1;
> 514	                ret =3D -ret;
> 515	            }
> 516	            env->gpr[3] =3D ret;
> 517	            break;

target/ppc/translate.c has :

static inline void gen_update_nip(DisasContext *ctx, target_ulong nip)
{
    if (NARROW_MODE(ctx)) {
        nip =3D (uint32_t)nip;
    }
    tcg_gen_movi_tl(cpu_nip, nip);
}
   =20
static void gen_exception_err(DisasContext *ctx, uint32_t excp, uint32_t =
error)
{
    TCGv_i32 t0, t1;
=20
    /* These are all synchronous exceptions, we set the PC back to
     * the faulting instruction
     */
    if (ctx->exception =3D=3D POWERPC_EXCP_NONE) {
        gen_update_nip(ctx, ctx->nip - 4);
    }
    t0 =3D tcg_const_i32(excp);
    t1 =3D tcg_const_i32(error);
    gen_helper_raise_exception_err(cpu_env, t0, t1);
    tcg_temp_free_i32(t0);
    tcg_temp_free_i32(t1);
    ctx->exception =3D (excp);
}
. . .
#if defined(CONFIG_USER_ONLY)
#define POWERPC_SYSCALL POWERPC_EXCP_SYSCALL_USER
#else
#define POWERPC_SYSCALL POWERPC_EXCP_SYSCALL
#endif
static void gen_sc(DisasContext *ctx)
{
    uint32_t lev;

    lev =3D (ctx->opcode >> 5) & 0x7F;
    gen_exception_err(ctx, POWERPC_SYSCALL, lev);
}


And there is:

Thread 1 hit Breakpoint 10, gen_sc (ctx=3D0x7ffffffe3f48) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/translate.c:3703
3703	    lev =3D (ctx->opcode >> 5) & 0x7F;
(gdb) bt
#0  gen_sc (ctx=3D0x7ffffffe3f48) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/translate.c:3703
#1  0x0000000060064f4b in gen_intermediate_code (env=3D0x860ea3ac0, =
tb=3D0x60723280 <static_code_gen_buffer+17296>)
    at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/translate.c:7360
#2  0x000000006003b090 in tb_gen_code (cpu=3D<optimized out>, =
pc=3D276092800, cs_base=3D0, flags=3D33579008, cflags=3D0)
    at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/translate-all.c:1276
#3  0x0000000060039c14 in tb_find (cpu=3D<optimized out>, =
last_tb=3D<optimized out>, tb_exit=3D<optimized out>)
    at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:363
#4  cpu_exec (cpu=3D<optimized out>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:671
#5  0x000000006003c988 in target_cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/ppc/target_arch_cpu.h:139
#6  cpu_loop (env=3D0x860ea3ac0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/main.c:121
#7  0x000000006003e003 in main (argc=3D<optimized out>, argv=3D<optimized =
out>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/b=
sd-user/main.c:516(gdb) finish
Run till exit from #0  gen_sc (ctx=3D0x7ffffffe3f48) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/translate.c:3703
gen_intermediate_code (env=3D0x860ea3ac0, tb=3D0x60723280 =
<static_code_gen_buffer+17296>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/translate.c:7365
7365	        if (unlikely(ctx.singlestep_enabled & CPU_SINGLE_STEP &&
(gdb) finish
Run till exit from #0  gen_sc (ctx=3D0x7ffffffe3f48) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/translate.c:3703
gen_intermediate_code (env=3D0x860ea3ac0, tb=3D0x60723280 =
<static_code_gen_buffer+17296>) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/translate.c:7365
7365	        if (unlikely(ctx.singlestep_enabled & CPU_SINGLE_STEP &&
(gdb) finish
Run till exit from #0  gen_intermediate_code (env=3D0x860ea3ac0, =
tb=3D0x60723280 <static_code_gen_buffer+17296>)
    at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/t=
arget/ppc/translate.c:7365
tb_gen_code (cpu=3D<optimized out>, pc=3D276092800, cs_base=3D0, =
flags=3D33579008, cflags=3D0) at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/translate-all.c:1277
1277	    tcg_ctx.cpu =3D NULL;
(gdb) finish
Run till exit from #0  tb_gen_code (cpu=3D<optimized out>, pc=3D276092800,=
 cs_base=3D0, flags=3D33579008, cflags=3D0)
    at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/translate-all.c:1277
0x0000000060039c14 in tb_find (cpu=3D<optimized out>, last_tb=3D<optimized=
 out>, tb_exit=3D<optimized out>)
    at =
/wrkdirs/usr/ports/emulators/qemu-user-static/work/qemu-bsd-user-17977d0/a=
ccel/tcg/cpu-exec.c:363
363	                tb =3D tb_gen_code(cpu, pc, cs_base, flags, 0);
Value returned is $23 =3D (TranslationBlock *) 0x60723280 =
<static_code_gen_buffer+17296>

Note that the:

 0x607233cc <static_code_gen_buffer+17628>:	movq   =
$0x1074d784,0x3c8(%r14)

is the emulated PC being forced to point back to the same syscall
instruction again, generated via gen_update_nip(ctx, ctx->nip - 4) .

The gen_sc instance (the first) seems to be responsible for:

 0x607233cc <static_code_gen_buffer+17628>:	movq   =
$0x1074d784,0x3c8(%r14)
 0x607233d7 <static_code_gen_buffer+17639>:	mov    %r14,%rdi
 0x607233da <static_code_gen_buffer+17642>:	mov    $0x203,%esi
 0x607233df <static_code_gen_buffer+17647>:	xor    %edx,%edx
 0x607233e1 <static_code_gen_buffer+17649>:	callq  0x600e0ed0 =
<helper_raise_exception_err>
=3D> 0x607233e6 <static_code_gen_buffer+17654>:	jmpq   0x6071ef06 =
<static_code_gen_buffer+22>
 0x607233eb <static_code_gen_buffer+17659>:	mov    $0x60723343,%eax
 0x607233f0 <static_code_gen_buffer+17664>:	jmpq   0x6071ef08 =
<static_code_gen_buffer+24>

being generated as its callers "finish" (see above).

(helper_raise_exception_err leads to siglongjmp so the
call does not return.)


If I interpret everything right, even for a successful
readlink (or other syscall) the emulated PC would not
be updated to the next instruction and the code would
loop, repeating the syscall.

It appears to me that something is wrong with the
logic:

static void gen_exception_err(DisasContext *ctx, uint32_t excp, uint32_t =
error)
{
    TCGv_i32 t0, t1;
=20
    /* These are all synchronous exceptions, we set the PC back to
     * the faulting instruction
     */
    if (ctx->exception =3D=3D POWERPC_EXCP_NONE) {
        gen_update_nip(ctx, ctx->nip - 4);
    }
. . .

or some balancing action is needed later for the likes of
readlink once it has completed the (in this case):

 ret =3D get_errno(readlink(path(p1), p2, arg3));

In fact for the early return (the -14 case) that
avoids the call t readlink the loop happens as
stands:

 LOCK_PATH(p1, arg1);
 p2 =3D lock_user(VERIFY_WRITE, arg2, arg3, 0);
 if (p2 =3D=3D NULL) {
     UNLOCK_PATH(p1, arg1);
     return -TARGET_EFAULT;
 }

So either the emulated PC should progress
for the early return or some other handling
should be involved for it: neither continuing
to the next instruction nor repeating the
instruction would seem appropriate. I expect
that this is a separate issue from the
readlink-used case as far as correct handling
goes.


=3D=3D=3D
Mark Millard
markmi at dsl-only.net




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?32379D16-A06E-4BE8-8FC5-C68A8B80E1D2>