Date: Fri, 10 Apr 2015 23:18:26 -0400 From: Philip Gollucci <pgollucci@gocurb.com> To: ruby@freebsd.org, ports@freebsd.org Cc: Matthew Vincenz <mvincenz@taximagic.com>, Scott Sanders <jssanders@gmail.com>, Stephen Garlick <sgarlick@taximagic.com> Subject: OPENSSL_ia32cap is my worst night mare Message-ID: <CADDV2kcb7828huyhV=vO8P5RMPHr2zC3Uy4qYOw1Y812e38aVA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
All,
This definitely needs an official bug report, I'm just not sure where.
So for those of you that don't know I work at a ruby on rails shop and we
also use AWS. We're just finishing our 10.x update. So take it as a given
all is well on 9.x.
- Spawn cperciva official AMI for FreeBSD 10.1-RELEASE
- Build and install ruby with ruby-install (gcc or clang). As far as I
can tell any version will do, but we're using 1.9.3-p551, 2.1.4, 2.2.1
- deploy a ruby on rails app with bundler/unicorn. any version of
bundler/unicorn seems to cause it.
- During unicorn start if you hook lib/libc/stdlib/getenv.c [patched
file attached]. You'll see an ENV request for OPENSSL_ia32cap. This
CORRUPTS the ENV. SMACK MY FACE AND PUNCH ME THEN HURT ME THEN CRY.
- Once corrupted the next thing in *that* process/sub-process to SET an
ENV var or rebuild the ENV will trigger it.
static const char CorruptEnvFindMsg[] = "environment corrupt; unable to
> find ";
which obviously causes the process to exit.
So as far as I can tell this is 100% reproducible. I must be the only
person running rails with unicorn in aws on fbsd.
So the good news is if you do
> export OPENSSL_ia32cap=' ';
You can hack around it.
I will provide any other information on demand. I'll even make you an AMI
if you want. Do note, that I can do a full buildworld, kernel, install, run
chef, install pkgs, run ruby and nothing breaks.
$ sudo kill -USR2 `cat pid of unicorn master`
is a good way to cause it.
The error msg is very misleading, I promise you BUNDLE_BIN_PATH has nothing
to do with it which is the ENV its trying to dick with but its CORRUPT.
If you apply my getenv.c file you'll see pointers/memory go everywhere.
reaped #<Process::Status: pid 97758 exit 0> worker=unknown
> executing ["/u/apps/rc/shared/vendor_bundle/ruby/1.9.1/bin/unicorn", "-E",
> "dev", "-c", "/usr/local/etc/unicorn/rc.rb", "-D",
> {23=>#<Kgio::TCPServer:fd 23>}] (in /u/apps/rc/releases/20150405023358)
> ruby: environment corrupt; missing value for BUNDLE_B~A^B
> /opt/rubies/1.9.3-p551/lib/ruby/gems/1.9.1/gems/bundler-1.9.2/lib/bundler/gem_path_manipulation.rb:5:in
> `[]=': Bad address - setenv (Errno::EFAULT)
> from
> /opt/rubies/1.9.3-p551/lib/ruby/gems/1.9.1/gems/bundler-1.9.2/lib/bundler/gem_path_manipulation.rb:5:in
> `preserve_gem_path'
> from
> /opt/rubies/1.9.3-p551/lib/ruby/gems/1.9.1/gems/bundler-1.9.2/lib/bundler.rb:12:in
> `<module:Bundler>'
> from
> /opt/rubies/1.9.3-p551/lib/ruby/gems/1.9.1/gems/bundler-1.9.2/lib/bundler.rb:11:in
> `<top (required)>'
> from
> /opt/rubies/1.9.3-p551/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in
> `require'
> from
> /opt/rubies/1.9.3-p551/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in
> `require'
> from
> /opt/rubies/1.9.3-p551/lib/ruby/gems/1.9.1/gems/bundler-1.9.2/lib/bundler/setup.rb:4:in
> `<top (required)>'
> from
> /opt/rubies/1.9.3-p551/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in
> `require'
> from
> /opt/rubies/1.9.3-p551/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in
> `require'
That *only* took 5 8 hour days of pain.
--
*PHILIP M. GOLLUCCI*
Sr. Director of IT | *Curb*
*w.* 703-579-6947
*m.* 703-336-9354
@gocurb <http://twitter.com/gocurb>;
Enter code* p6magic* for $10 off your first ride
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CADDV2kcb7828huyhV=vO8P5RMPHr2zC3Uy4qYOw1Y812e38aVA>