Ensure single quotes are used for string literals.
SQLite 3.27.0 introduced a change in behaviour and started issuing
warnings to the error log when double quotes were being used for string
literals. This happens to have been the way pkgin would insert every
package into the database, causing huge disk I/O to the sql.log file
when running "pkgin update".
At best this was significantly impacting runtime by orders of magnitude
and wasting disk space with large sql.log files, and at worst this was
reproducibly causing updates to fail entirely with "Short read of
pkg_summary: Lzma library error: No progress is possible".
The exact details of why the extra logging was causing lzma failures is
still unclear and needs to be looked into to ensure we aren't missing
something else, but this change is demonstrated to avoid the issue.
Revert unsafe database pragmas.
There appears to be no discernable performance impact when prioritising
safety over speed, and should help to avoid issues seen by users where
the database becomes corrupted if they CTRL-C during operations.
Fix -Werror=conversion failure in trimcr().
While here avoid a separate strlen call in read_repos(), using the
length returned by trimcr(). The check for a NULL buf is removed as it
was already tested earlier in the function.
Fix handling of unavailable and preferred packages.
During some of the earlier rototilling, the package not available
message was no longer printed for unavailable packages due to changes in
the download code. This is now fixed, and also differentiates correctly
between packages that are not available, and those that do not match the
requirements stated in preferred.conf.
Use separate functions for install and show-* commands, the former using
the preferred.conf logic, the latter using regular matches to match
local queries and other tools. This also avoids "not preferred"
messages leaking into show-* output, they belong in trace/debug output.
Abstract away some of the logic that will help in future cleanups, and
improve the output messages. Upgrades still do not support preferred
matches, this will be fixed in due course.
Add DEPENDS matching for the full dependency tree.
Previously pkgin would simply consider the latest available version of a
package and recurse into its dependency tree, but this could lead to
issues if it was later found that the latest version is not suitable for
This was shown most clearly by conflicts between net/samba (3.x) and
net/samba4 (4.x). Packages that pulled in net/samba and a DEPENDS of
samba>=3.x<4.0 would eventually fail when trying to install devel/talloc
(a dependency of net/samba4) alongside net/samba as those two packages
both try to install include/talloc.h.
By correctly matching DEPENDS during the full tree scan, net/samba4 is
no longer considered and devel/talloc is no longer pulled in as a
The implementation isn't ideal, and eventually will be replaced with
better logic, but the correct behaviour is desired first (along with
appropriate test cases), improvements second.
Allow -y flag to work with autoremove.
The previous behaviour violated the principle of least surprise for no
apparent reason. If anything, autoremove is less dangerous than
install, remove, or upgrade - all of which support -y.
Use full package name when importing via PKGPATH.
This ensures that, where multiple versions exist, we pick the correct
one. Previously, an import of e.g. lang/nodejs6 would try to install
whatever the latest version available was.
Show pkg_info output inline instead of buffering.
This avoids an occasional issue where the exec_list() output was being
corrupted. Buffering the entire output could be considered useful if we
were actually using it to return an error status and print a useful
message, but we weren't, so this way is faster, uses less memory, and
allows us to remove more code.
We also now correctly return the exit code of pkg_info.
Only consider a refresh if PKGPATH matches.
Fixes an issue where "pkgin upgrade" considered nodejs-8 a refresh of
nodejs-6 when calculating it as a dependency of yarn, whereas "pkgin
full-upgrade" did not.
Permit the user to install an older pkg version.
For now this counts as an upgrade, even though the version is going
backwards. In the future this will probably be correctly accounted as a
downgrade action or similar.
Downgrades are only supported for packages that are specifically
requested on the command line, e.g. "pkgin in nodejs-6", otherwise they
are ignored, which was previously the behaviour for all packages.
Ensure warn/err counters are reset for each phase.
This is a temporary fix. The logfile code uses far too many global
variables, and really should combine the results at the end of the run
to avoid an earlier phase with errors possibly getting lost.
While here remove an unused global variable and an incorrect function
Improve the output action format.
If able, format the list of packages to the current window width, and
indent slightly to make a lot more readable, especially with large
numbers of packages.
For -n retain one package per line.
Add support for displaying packages to be refreshed.
As there is now a split between the actions and there is no total for
the install count, add a separate summary at the end displaying the
totals of each action along with the space requirements.
Stop abusing struct Pkglist for different types.
Get rid of the different types and just have a single Pkglist with a
standard set of entries. Not only does this simplify things, it avoids
a nasty shadowing issue where Pkglist.old was both an entry and a define
into a different union member.
At some point in the future this will be split up properly into structs
which only have exactly what they need.
Add initial implementation of in-place upgrades.
Instead of performing a remove then reinstall of every package marked
for upgrade, just use "pkg_add -DU" to upgrade in-place. This has a
number of benefits:
* Preserved packages are handled correctly and are now upgraded without
* Ordering issues no longer cause "package is already installed"
* It is faster.
There are some cases where this isn't optimal yet and can still fail,
for example when trying to handle conflicts where we do legitimately
need to remove packages first before re-installing newer versions. This
previously wasn't handled correctly either, so it is not considered a
regression, and will be improved in due course.
Add a separate download package list and counters.
This helps to extract some of the logic away from the install list,
paving the way for handling upgrades and refreshes correctly, and
improving the download-only messages.
Remove some messages that aren't useful.
The "symlinking ..." message provides no real value, and can confuse
users when upgrades include BUILD_DATE refreshes, as the number of
messages may not match the number of packages to be acted upon if
symlinks already exist from a previous run.
The "installing packages..." message isn't very useful as immediately
after that we print a separate "installing..." message for each package.
The "nothing to upgrade" and "nothing to install" messages are of no use
either, and again can be potentially confusing, for example if all
you've requested is to perform downloads of packages.
While here remove some debug code to make pkgin_install a little more
Fix -f flag when used with install.
Previously the remote databases would be wiped but not reinserted
correctly due to confusion with the various global variables handling
this logic. Fix the issue and simplify things a little until the
repository initialisation code can be overhauled correctly.
Fix refresh upgrades for preserved packages.
Using pkg_add -ffu is not adequate if the version is identical, pkg_add
will not reinstall the package. We need to use -ffU to force pkg_add to
accept the refresh.
This is merely a stop-gap fix. Preserve package upgrades are still
handled terribly by ignoring the pkg_delete errors, and will be fixed
properly in due course.
Improve provides/requires messages.
The most common confusion for users is that they believe provides and
requires refer to the full list of files provided or required by the
package, when in reality they only refer to shared libraries.
Overhaul SQL error logging.
Use the primary sqlite3 interface for error logging, using a callback
function to report any errors as they happen. This will allow us to
interface directly to the database if necessary rather than having to go
via the pkgindb_doquery() interface, and also provides useful error
messages rather than the very unhelpful "callback aborted".
Update the log output to include the date, format better for
readability, and open in append mode so we don't potentially lose useful
While here reorganise some functions into logical order and move the
remaining function declarations to pkgin.h
Don't change MSG_CALCULATING_DEPS based on DEBUG.
Just use the istty check to determine whether to use the spinner or not,
helps to avoid control characters in non-terminal output, most notably
the test suite.
Simplify and improve "pkgin stats".
Use the sqlite3_* interfaces directly rather than going via the
pkgindb_doquery() wrapper. This allows values to be fetched directly
rather than having to be converted, and also enables 4 queries to be
reduced to just one.
Simplify and inline the output rather than having a bunch of defines for