From owner-freebsd-current@FreeBSD.ORG Fri Jun 17 18:02:33 2005 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3473E16A41C for ; Fri, 17 Jun 2005 18:02:33 +0000 (GMT) (envelope-from peadar@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E38F43D49 for ; Fri, 17 Jun 2005 18:02:33 +0000 (GMT) (envelope-from peadar@FreeBSD.org) Received: from freefall.freebsd.org (peadar@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.3/8.13.3) with ESMTP id j5HI2WOR026423 for ; Fri, 17 Jun 2005 18:02:32 GMT (envelope-from peadar@freefall.freebsd.org) Received: (from peadar@localhost) by freefall.freebsd.org (8.13.3/8.13.1/Submit) id j5HI2WBS026422 for current@freebsd.org; Fri, 17 Jun 2005 18:02:32 GMT (envelope-from peadar) Date: Fri, 17 Jun 2005 18:02:32 +0000 From: Peter Edwards To: current@freebsd.org Message-ID: <20050617180232.GA25818@freefall.freebsd.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="HlL+5n6rz5pIUxbD" Content-Disposition: inline User-Agent: Mutt/1.4.2.1i Cc: Subject: Towards a working "wine". [long] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jun 2005 18:02:33 -0000 --HlL+5n6rz5pIUxbD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Y'all, I wanted to run a substantial windows app using "wine", and failed miserably. Looking around, it appears to be an acknowledged fact that wine is borked on FreeBSD, so I did some hacking about. The problems I found are centered around address space allocation: Wine is inevitably somewhat sensitive to address space layout, seeing as it has to present the Win32 binaries with a reasonably familiar environment. (All this assumes standard kernel, etc. Fixed addresses are the order of the day) Problem 1: When starting up, Wine needs to mmap stuff at address 0x80000000, but does so without using MAP_FIXED: I think the intention is that the code involved should be "best effort" to map to the hint address supplied to mmap(), rather than an all-or-nothing thing. FreeBSD's mmap sees this as an address in the data segment (see problem 2 below), and shifts the hint along to an address past there. There appears to be a MAP_TRYFIXED flag that wine uses on some systems that affords the hint more weight, which is pretty trivial to implement (see wine_mmap.txt). That was enough to get my app running, but not for long. Problem 2: The "wine" executable itself is located strangely, so the text and data go right up near the 2G point to help out Win32. (actually at address 0x77f00000, or 2GB-129MB). This is causes the heap to appear at such a high address, as FreeBSD merges the data from the executable image with the heap space (which malloc uses). Once the mmap at 0x80000000 happens, that puts a nice hole in the data segment that means sbrk starts to fail after 128MB is allocated. (On linux, it appears that the brk point and load address of the data segment from the executable are unrelated, and malloc() pays little attention to [s]brk() anyhow.) End result is, that with the break point sitting where it does, and the mapped Windows stuff 129MB above it, wine is very low on malloc()able memory. There is a disasterously ugly hack attached, wine_malloc.txt that hacks on malloc(), and adds a "W" option to enable the hack. This works by trading the brk()/sbrk() calls for an mmapping starting at 0xa0000000, which should be able to grow towards the process stack. (phkmalloc works with a large contiguous heap, rather than a fragmented one, so a more "pure" mmap-based approach won't fit into it too smoothly.) WARNING: Don't put the malloc patch directly into libc unless you are in the mood to deal with a hosed libc. I ran it like this: > $ cd /tmp > $ cp /usr/src/lib/libc/stdlib/malloc.c ./wine_malloc.c > $ patch wine_malloc.c < /tmp/wine_malloc.txt > $ cc -I /usr/src/lib/libc/include -o wine_malloc.so \ > -shared wine_malloc.c Then invoke wine as $ env LD_PRELOAD=/tmp/wine_malloc.so MALLOC_OPTIONS=W wine ... With these patches, wine is much, much more stable for me. I'd like to commit the kernel part at some stage if someone can review it, but the malloc() part seems like a particularly ugly hack. It could be cleaned up and added as a patch to the port I suppose, but has anyone got any better ideas? --HlL+5n6rz5pIUxbD Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="wine_malloc.txt" --- A.c Fri Jun 17 18:04:15 2005 +++ B.c Fri Jun 17 18:20:54 2005 @@ -285,6 +285,51 @@ static void ifree(void *ptr); static void *irealloc(void *ptr, size_t size); +/* + * Ugly Wine compatibility stuff: + * This trades brk/sbrk in the data segment for mmap() from WINE_HEAPBASE up + * to the process's stack. + */ + +static caddr_t heapbase; +static caddr_t heaptail; +#define WINE_HEAPBASE ((caddr_t)0xa0000000) + +static caddr_t +heapend() +{ + if (!heapbase) + return sbrk(0); + return heaptail; +} + +static int +heapset(caddr_t point) +{ + caddr_t end; + int error; + + if (!heapbase) + return brk(point); + + /* Page-align the current end point */ + end = (caddr_t)((intptr_t)(heaptail + malloc_pagesize - 1) & + ~(malloc_pagesize - 1)); + error = 0; + + /* Grow or shrink? */ + if (end > point) { + error = munmap(point, end - point); + } + if (end < point) { + error = mmap(end, point - heaptail, PROT_READ|PROT_WRITE, + MAP_ANON|MAP_PRIVATE, -1, 0) == MAP_FAILED ? -1 : 0; + } + if (!error) + heaptail = point; + return error; +} + static void wrtmessage(const char *p1, const char *p2, const char *p3, const char *p4) { @@ -328,12 +373,12 @@ { caddr_t result, tail; - result = (caddr_t)pageround((u_long)sbrk(0)); + result = (caddr_t)pageround((u_long)heapend()); tail = result + (pages << malloc_pageshift); if (tail < result) return (NULL); - if (brk(tail)) { + if (heapset(tail)) { #ifdef MALLOC_EXTRA_SANITY wrterror("(ES): map_pages fails\n"); #endif /* MALLOC_EXTRA_SANITY */ @@ -457,6 +502,7 @@ case 'X': malloc_xmalloc = 1; break; case 'z': malloc_zero = 0; break; case 'Z': malloc_zero = 1; break; + case 'W': heapbase = heaptail = WINE_HEAPBASE; break; default: _malloc_message(_getprogname(), malloc_func, " warning: ", "unknown char in MALLOC_OPTIONS\n"); @@ -485,7 +531,7 @@ * We need a maximum of malloc_pageshift buckets, steal these from the * front of the page_directory; */ - malloc_origo = ((u_long)pageround((u_long)sbrk(0))) >> malloc_pageshift; + malloc_origo = ((u_long)pageround((u_long)heapend())) >> malloc_pageshift; malloc_origo -= malloc_pageshift; malloc_ninfo = malloc_pagesize / sizeof *page_dir; @@ -532,7 +578,7 @@ wrterror("(ES): zero entry on free_list\n"); if (pf->page > pf->end) wrterror("(ES): sick entry on free_list\n"); - if ((void*)pf->page >= (void*)sbrk(0)) + if ((void*)pf->page >= (void*)heapend()) wrterror("(ES): entry on free_list past brk\n"); if (page_dir[ptr2index(pf->page)] != MALLOC_FREE) wrterror("(ES): non-free first page on free-list\n"); @@ -948,7 +994,7 @@ if (pf->next == NULL && /* If we're the last one, */ pf->size > malloc_cache && /* ..and the cache is full, */ pf->end == malloc_brk && /* ..and none behind us, */ - malloc_brk == sbrk(0)) { /* ..and it's OK to do... */ + malloc_brk == heapend()) { /* ..and it's OK to do... */ /* * Keep the cache intact. Notice that the '>' above guarantees that @@ -957,7 +1003,7 @@ pf->end = (char *)pf->page + malloc_cache; pf->size = malloc_cache; - brk(pf->end); + heapset(pf->end); malloc_brk = pf->end; index = ptr2index(pf->end); --HlL+5n6rz5pIUxbD Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="wine_mmap.txt" Index: sys/sys/mman.h =================================================================== RCS file: /usr/cvs/FreeBSD/src/sys/sys/mman.h,v retrieving revision 1.40 diff -u -r1.40 mman.h --- sys/sys/mman.h 2 Apr 2005 12:33:31 -0000 1.40 +++ sys/sys/mman.h 16 Jun 2005 15:20:32 -0000 @@ -87,6 +87,7 @@ * Extended flags */ #define MAP_NOCORE 0x00020000 /* dont include these pages in a coredump */ +#define MAP_TRYFIXED 0x00040000 /* take hint more seriously */ #endif /* __BSD_VISIBLE */ #if __POSIX_VISIBLE >= 199309 Index: sys/vm/vm_mmap.c =================================================================== RCS file: /usr/cvs/FreeBSD/src/sys/vm/vm_mmap.c,v retrieving revision 1.200 diff -u -r1.200 vm_mmap.c --- sys/vm/vm_mmap.c 14 Apr 2005 16:03:30 -0000 1.200 +++ sys/vm/vm_mmap.c 17 Jun 2005 17:34:32 -0000 @@ -256,6 +256,8 @@ addr -= pageoff; if (addr & PAGE_MASK) return (EINVAL); + } + if (flags & (MAP_FIXED | MAP_TRYFIXED)) { /* Address range must be all in user VM space. */ if (addr < vm_map_min(&vms->vm_map) || addr + size > vm_map_max(&vms->vm_map)) --HlL+5n6rz5pIUxbD--