From owner-freebsd-threads@FreeBSD.ORG Sun Jun 1 15:10:52 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D4E6637B401 for ; Sun, 1 Jun 2003 15:10:52 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.bayarea.net [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id E215143F75 for ; Sun, 1 Jun 2003 15:10:51 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h51MApwk096096 for ; Sun, 1 Jun 2003 15:10:51 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h51MAp2Y001042 for ; Sun, 1 Jun 2003 15:10:51 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h51MAptR001041 for threads@FreeBSD.org; Sun, 1 Jun 2003 15:10:51 -0700 (PDT) Date: Sun, 1 Jun 2003 15:10:51 -0700 From: Marcel Moolenaar To: threads@FreeBSD.org Message-ID: <20030601221051.GA975@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.4i Subject: libthr: ia64 port: trapframe issue X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2003 22:10:53 -0000 Gang, After implementing cpu_thread_setup() as Mike pointed out I hit upon another snafu. In kern_thr.c:thr_create() we copy the trapframe over from the currently running thread (td) to the newly created thread (td0). This however cannot be done on ia64: On ia64 we can have a nested fault when trying to contruct a trapframe. A nested fault is icky, because the CPU does not save any state (it would otherwise clobber the state of the original trap/interrupt). This also means that we have to know exactly what can cause a nested fault so that we know what the faulting address is and where we have to jump to when we inserted a TLB. This means that we have to make sure that a trapframe does not cross a page boundary, because otherwise we can have a nested fault any place we write into the trapframe. To solve this, we align the trapframe on a 1KB boundary. Now the crux: if we align the trapframe, we don't know how much the delta is from the previous stack pointer to the bottom of the trapframe and hence the new stack pointer. This itself is not really a problem, but we need to know this delta if we want to restore the stack pointer to the value it had prior to the trap (we're talking kernel stack pointer). Therefore, we save the length of the trapframe (ie the delta) in the trapframe itself. By copying trapframes, we also copy the length of the trapframe, which is generally bad. The new trapframe is not 1KB aligned (we know up front it will be contained in a single page) and thus will have a fixed length (sizeof(struct trapframe)). However the trapframe we copy from has been 1KB aligned and is generally larger than sizeof(trapframe). Anyway: the end result is that if we enter userland with a thread created by thr_create, we clobber our PCB because we incorrectly updated the stack pointer. The following solutions exist: 1. quick and dirty: ------------------- We know that the new trapframe is sizeof(struct trapframe) long, so we simply correct the length after copying: Index: kern_thr.c =================================================================== RCS file: /home/ncvs/src/sys/kern/kern_thr.c,v retrieving revision 1.8 diff -u -r1.8 kern_thr.c --- kern_thr.c 16 May 2003 21:26:42 -0000 1.8 +++ kern_thr.c 1 Jun 2003 22:02:46 -0000 @@ -157,6 +157,9 @@ td0->td_sigmask = td->td_sigmask; PROC_UNLOCK(td->td_proc); bcopy(td->td_frame, td0->td_frame, sizeof(struct trapframe)); +#ifdef __ia64__ + td0->td_frame->tf_length = sizeof(struct trapframe); +#endif td0->td_ucred = crhold(td->td_ucred); /* Initialize our kse structure. */ 2. cpu__set_upcall() change: ---------------------------- We have all the MD specifics in cpu_set_upcall, so we move the copying of the trapframe to cpu_set_upcall(). This means we need to change its prototype to either void cpu_set_upcall(struct thread *new, struct thread *old) or void cpu_set_upcall(struct thread *new, struct pcb *pcb, struct trapframe *tf) I don't know if we can simply remove the copying of the trapframe and have it happen after crhold() and kse_alloc() or that we need to move the call to cpu_set_upcall() up to before kse_alloc() and/or crhold(). In other words: I don't know if we have a phase ordering problem if we delay copying the trapframe until we call cpu_set_upcall() Thought? -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net