153 Commits

Author SHA1 Message Date
thfr 6eb73f0604 Fix constraints on _DistTuple columns. This makes the DistTuple entries
in sqlports complete. (Before only the last set was shown.)

From espie, instigated by a less correct diff from me. Also feedback
from sthen@ ("makes a lot of sense")
2025-12-01 00:30:35 +00:00
sthen a76114ab0d don't rely on the sqlite3 cli accepting double-quoted strings as literals
if they don't match a valid identifier, as of 3.41.0 (2023-02-21) standard
builds of the cli no longer accept this and warn instead. currently builds
from the "autoconf" distribution file do still accept this, but sqlports
should not rely on something that upstream are trying to teach people not
to use.  https://sqlite.org/quirks.html#dblquote

from espie
2025-11-03 14:46:04 +00:00
espie d6b51a6e93 better tracking of paths we should do.
got a bit lost in pkg equivalence, but now it will report
RUN_DEPENDS = devel/llvm
as a problem, as expected
2023-11-26 18:05:09 +00:00
espie d7fc0908f8 actually reference UNLINKED 2023-11-11 11:53:07 +00:00
espie 9405055eec major overhaul of the treewalker and options for mksqlitedb
as a result:
- properly show errors at the end.
- show walker errors like I did for dpb.
- feature a "keepgoing" mode that will do the full walk BEFORE exiting
with an error if necessary (but switch from +++ path to !!! path after
the first error, so that interactive runs can be interrupted early on
if deemed applicable)
- feature a "testmode" for trying out a single path
- remove some old obsolete options
- embed some perldoc in the script proper.
2023-10-25 15:23:59 +00:00
espie b09d4bd62b stop special-casing MASTER_SITES0..9 since it no longer exists 2023-09-25 17:10:44 +00:00
espie 16a4e0177b register ROACH_URL and ROACH_SITES 2023-09-06 21:07:16 +00:00
espie d48dda1abf go back to naming it 'MasterSites' for now so that portroach is less
confused.

(todo: emit views to all distfiles and all mastersites to help the poor thing)
2023-09-06 11:56:34 +00:00
espie a22a529118 convert to SITES 2023-09-05 13:51:33 +00:00
espie 78c5b8ae8c add support for DIST_TUPLE* 2023-09-03 11:40:30 +00:00
espie 3f2e002cf0 register EXTRACT_FILES as well 2023-08-31 13:49:22 +00:00
espie 8775a88e13 document sufx in _distfiles, add n to the view.
add infrastructure to save MOD* values.
2023-08-22 14:59:46 +00:00
espie da9e6ab375 documentation should match reality 2023-08-21 11:35:19 +00:00
espie c581ab1d2c fix thinko 2023-08-21 10:10:01 +00:00
espie b6f7e25201 support for DISTFILES/SUPDISTFILES/PATCHFILES.sufx and MASTER_SITES.sufx
note that only the non suffixed versions end up in the main ports view.
2023-08-14 17:36:43 +00:00
espie 73a357a27d somehow I forgot about patchfiles in the manpage. 2023-08-08 11:12:06 +00:00
sthen 2e8b030b2e Support USE_NOBTCFI in ports to add -z nobtcfi to linker command lines and
record this information in the sqlports db.  ok kn@ tb@

OpenBSD is starting to support branch target identification on amd64 and
arm64 (part of the features on Apple m2, and on Intel 11th gen/newer CPUs
with "control-flow enhancement technology").

On amd64 it is currently being enabled/disabled in snapshot kernels at
various times while we gain more information about which software in
ports is working/not.

This works by placing certain opcodes at legitimate targets of branch
instructions (which are ignored on earlier CPUs as they are NOPs there)
and trapping if an indirect call/jump is attempted to a location which
does not contain such an opcode. This makes it harder for an attacker
to jump to a location containing code of their choosing.

For more details on the Intel implementatios See chapter 17 of SDM vol.1
https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html

Our system compilers on amd64/arm64 add the required opcodes at valid
targets, but some code in ports is built with a compiler which does not
do that, also some ports have asm code which has not had the required
instructions added at those points yet (i.e. endbr64 on amd64).
In those cases, the whole binary can be annotated with a segment type
PT_OPENBSD_NOBTCFI as an indicator to the kernel not to enforce branch
target control flow integrity for the produced binary.

The relevant compiler options to enable (default on OpenBSD)/disable are:

amd64: -fcf-protection=branch, -fcf-protection=none
arm64: -mbranch-protection=bti, -mbranch-protection=none
2023-06-12 10:06:29 +00:00
sthen 8944205b73 record USE_NOEXECONLY in sqlports 2023-01-09 17:09:55 +00:00
naddy 7a958849ed drop RCS Ids 2022-03-11 18:31:17 +00:00
ajacoutot 32baf14873 Unbreak COMMENT-main and bump. 2021-11-23 07:10:49 +00:00
espie 737fb8bb76 zap a few PKGNAME-sub which are now default
(and also duplicated EPOCH-sub which make little sense in db/{v3,v4}
2021-11-22 13:55:54 +00:00
espie de8d34ed61 add FIX_CRLF_FILES to secondary tables 2021-02-25 23:26:30 +00:00
espie 4d7cd6c96a document DEBUG_* tables and views 2021-02-21 09:14:05 +00:00
espie ec88bfaedb finally add DEBUG_PACKAGES and DEBUG_CONFIGURE_ARGS to sqlports
not sure what I missed the last time as this is fundamentally trivial
2021-02-20 18:05:04 +00:00
espie c10c81e8a7 simplify my debug process by allowing to restart build from a given point 2021-01-29 10:28:30 +00:00
espie c0b8252709 add the date at which the db was generated into sqlports.
document Meta table, which I somehow forgot
2021-01-29 08:30:37 +00:00
espie b95f372d1d flesh out some more stuff in the manpage.
do a REV bump for once, since we don't actually change any real details
2020-07-07 10:53:27 +00:00
espie f652d789b5 move description of the schema to a manpage 2020-07-06 09:06:45 +00:00
espie 2159135658 extend request a bit more so that verbose mode also shows type of dependency 2020-06-11 19:55:15 +00:00
espie a0cef780a2 verbose mode, now that I've figured it out again 2020-06-11 16:05:52 +00:00
espie f03a34da8b add a few options and a manpage to show-reverse-deps 2020-06-11 15:40:41 +00:00
espie 00fb740c85 handling of DEBUG_PACKAGES/DEBUG_CONFIGURE_ARGS is bogus
just ignore them for now
2020-05-19 08:50:55 +00:00
espie 2941d54083 add DEBUG_PACKAGES and DEBUG_CONFIGURE_ARGS
make sqlports strict by default (ignore PORTSDIR_PATH and look only in
PORTSDIR)
2019-11-11 20:44:39 +00:00
kn 870bcebea8 Say pkgpath not path in show-reverse-deps usage
Drop useless use of cat(1) and quote user input while here.

OK espie
2019-10-03 12:41:22 +00:00
espie 70d0d70b25 fix sqlports, bump to make sure 2019-07-20 23:12:47 +00:00
sthen 98e1189996 Add a belt-and-braces check for the number of lines output in the produced
ports-INDEX, error out if it's too small.
2019-07-18 06:35:43 +00:00
espie d45c8394c9 bye bye old PERMIT_* 2019-07-14 11:27:19 +00:00
sthen 9fe1e38b23 replace simple PERMIT_PACKAGE_CDROM=Yes with PERMIT_PACKAGE=Yes 2019-07-12 20:43:27 +00:00
espie 3ec73388de make sure to stop if scripts fail
as noticed by sthen@
2019-06-04 16:05:52 +00:00
espie 7271781869 phase out old PERMIT_* variables.
We won't be distributing CDROM anymore, so simplify to
just PERMIT_PACKAGE / PERMIT_DISTFILES

In particular, the new variables are shorter, so this makes
for better cosmetic sense in ports.

The "current" version allows for the old variables to die out
peacefully, at our leasure, and then I'll remove the old stubs.

As discussed with sthen@, various people agree, and deraadt@
is okay with the strategic change.

This does survive a bulk.

PLEASE NOTE: built packages require current pkg* tools.
In particular, the current version deals with
@comment pkgpath=* ftp=*
just fine, but the old one WILL COMPLAIN about missing cdrom info.

register-plist also doesn't care, adding/removing cdrom info
is a no-op for it. There is NO BUMP needed for the conversion.

For now, sqlports STILL carries the old variables. I'll deal
with their removal later.
2019-05-31 21:27:03 +00:00
espie 71a2ef4f62 Add a denormalized table that instantiates the main ports view.
So, Portsq is a snapshot of Ports... dirty but fast.

I had mixed feelings about this.
There's also a script to resync the table.

Grows the db by about 50% (+25MB)

Discussed with sthen@
2019-05-18 21:35:21 +00:00
espie 78fb9be676 create indices for all fullpkgpaths in secondary tables, this ought
to speed up some requests at least a little bit.
2019-05-17 20:41:54 +00:00
espie 9f70fcb4b8 Fix bug in argument order, strenghten argument parsing to avoid that.
Add a "meta" table that just contains a schema version and a hash.

So that some clients (e.g., portroach) can automatically figure out
whether a rebuild is required.
2019-03-15 11:29:53 +00:00
espie 86c6128347 slightly more correct subrequest (fullpkgpath is ordered "by accident")
I suspected this, confirmed after a discussion with Reda Dehak, from my
lab at work.
2019-03-11 13:29:21 +00:00
espie 4c1a6388c5 use "chained joins" to create canonical_depends with the Sql.pm framework
kill a bit of code.

adjust is now the only request "not in the mold", so just create it when
needed (so, late enough)

Add index creation (directly in create_schema), for now used for
canonical.
2019-01-22 16:55:22 +00:00
espie 8a522c95ec incorporate shared_libs into main view 2019-01-12 11:10:00 +00:00
espie 98537e4d7f put distfiles in its own table along with indexers 2019-01-11 21:52:25 +00:00
espie d906df9046 fix wantlib view to not create bogus entries
create a wantlib_ordered view with full lists of wantlib
re-incorporate wantlib into main ports view
2019-01-11 19:52:14 +00:00
espie e14a11decb fix the _wantlib table (it should separate >=10 as extra)
AND amend the wantlib view so that it has the "complete" wantlib
2019-01-11 17:17:13 +00:00
espie ee84de609c add descr/readme to ports view, simplify print-ports-index
don't left join on categories, it's always here.

make 'alter-tables' full

wantlib_ordered does not work yet... more stuff to look at.
2019-01-09 15:46:20 +00:00