From owner-freebsd-questions@freebsd.org  Sun May 17 03:26:53 2020
Return-Path: <owner-freebsd-questions@freebsd.org>
Delivered-To: freebsd-questions@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id C2CA213B873
 for <freebsd-questions@mailman.nyi.freebsd.org>;
 Sun, 17 May 2020 03:26:53 +0000 (UTC)
 (envelope-from rfg@tristatelogic.com)
Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com
 [69.62.255.118])
 by mx1.freebsd.org (Postfix) with ESMTP id 49Pncr63ZQz3bFL
 for <freebsd-questions@freebsd.org>; Sun, 17 May 2020 03:26:52 +0000 (UTC)
 (envelope-from rfg@tristatelogic.com)
Received: by segfault.tristatelogic.com (Postfix, from userid 1237)
 id E527B4E73E; Sat, 16 May 2020 20:26:51 -0700 (PDT)
From: "Ronald F. Guilmette" <rfg@tristatelogic.com>
To: Yuri Pankov <ypankov@fastmail.com>
cc: freebsd-questions@freebsd.org
Subject: Re: (character) Conversion error (in vi) ?
In-Reply-To: <910485e6-a5bf-8da1-55bb-2bc632d657e3@fastmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-ID: <72850.1589686011.1@segfault.tristatelogic.com>
Content-Transfer-Encoding: 8bit
Date: Sat, 16 May 2020 20:26:51 -0700
Message-ID: <72851.1589686011@segfault.tristatelogic.com>
X-Rspamd-Queue-Id: 49Pncr63ZQz3bFL
X-Spamd-Bar: /
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=pass (mx1.freebsd.org: domain of rfg@tristatelogic.com designates
 69.62.255.118 as permitted sender) smtp.mailfrom=rfg@tristatelogic.com
X-Spamd-Result: default: False [0.70 / 15.00]; ARC_NA(0.00)[];
 FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[];
 R_SPF_ALLOW(-0.20)[+mx:c]; MIME_GOOD(-0.10)[text/plain];
 DMARC_NA(0.00)[tristatelogic.com];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2];
 FREEMAIL_TO(0.00)[fastmail.com]; RCVD_COUNT_ZERO(0.00)[0];
 FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
 MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:14051, ipnet:69.62.128.0/17, country:US];
 SUBJECT_ENDS_QUESTION(1.00)[]
X-BeenThere: freebsd-questions@freebsd.org
X-Mailman-Version: 2.1.33
Precedence: list
List-Id: User questions <freebsd-questions.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-questions>, 
 <mailto:freebsd-questions-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-questions/>
List-Post: <mailto:freebsd-questions@freebsd.org>
List-Help: <mailto:freebsd-questions-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-questions>, 
 <mailto:freebsd-questions-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 17 May 2020 03:26:53 -0000

In message <910485e6-a5bf-8da1-55bb-2bc632d657e3@fastmail.com>, 
Yuri Pankov <ypankov@fastmail.com> wrote:

>> No, it's not that bug after all.  The issue is that (n)vi now (for quite 
>> some time :-) defaults to UTF-8 when it can't reliably detect the file 
>> encoding, so you'll just have to help it a bit adding the following to 
>> ~/.nexrc:
>> 
>> set fileencoding=iso8859-1
>> 
>> This way (n)vi will check if file encoding looks like UTF-8, and if not, 
>> it will use ISO8859-1 as fallback.
>
>For the sake of correctness, re-reading the code, my reply was not 
>entirely precise: (n)vi first checks if file is valid UTF-8, and if it 
>isn't and fallback encoding (as shown above) is not set, uses your 
>locale's encoding which I guess is UTF-8, hence failing.

I am far from knowledgable when it comes to content encoding &
character sets generally, but it would seem to me that if the
fallback is the same as the encoding that was already tried
and failed, then this simple-minded fallback strategy may
perhaps be sub-optimal.


Regards,
rfg