summaryrefslogtreecommitdiff
path: root/usr.bin/sort/files.c
AgeCommit message (Collapse)Author
2015-08-05add a description about what was being attempted to failed writes messages.mrg
2009-11-06Retire __SCCSID. It has only archeological value now. Also retire lintjoerg
conditional around __RCSID, lint can handle that fine.
2009-10-07long align records written to temporary files.dsl
2009-09-28Fix borked fix for sort relying on realloc() changing the buffer end.dsl
Sorts of more than 8MB data now probably work again.
2009-09-26Move all the fopen() calls out of the record read routines into the callers.dsl
Split the merge sort so that fsort() can pass the 'FILE *' of the temporary files to be merged into the merge code. Don't rely on realloc() not moving the end address of a buffer! Rework merge sort so that it sorts pointers to 'struct mfile' and only copies about sort record descriptors. No functional change intended.
2009-09-10Save length of key instead of relying of the weight of the record sep.dsl
This frees a byte value to use for 'end of key' (to correctly sort short keys) while still having a weight assigned to the field sep. (Unless -t is given, the field sep is in the field data.) Do reverse sorts by writing the output file in reverse order (rather than reversing the sort - apart from merges). All key compares are now unweighted. For 'sort -u' mark duplicates keys during the sort and don't write to the output. Use -S to mean a posix sort - where equal keys are sorted using the raw record (rather than being kept in the original order). For 'sort -f' (no keys) generate a key of the folded data (as for -n -i and -d), simplifies the code and allows a 'posix' sort.
2009-09-05Now we have our own radix_sort() change the interface so that we passdsl
an array of 'RECHEADER *' and remove all the crappy stuff that backed up by REC_DATA_OFFSET (etc). Also change radix_sort() to return the number of elements, soon to be used to drop duplicate keys (for sort -u).
2009-08-22Rework the way sort generates sort keys:dsl
- If we generate a key, it is always sortable using memcmp() - If we are sorting the whole record, then a weight-table must be used during compares. - Major surgery to encoding of numbers to ensure unique keys for equal numeric values. Reverse numerics are handled by inverting the sign. - Case folding (-f) is handled when the sort keys are generated. No other code has to care at all. - Key uniqueness (-u) is done during merge for large datasets. It only has to be done when writing the output file for small files. Since the file is in key order this is simple! Probably fixes all of: PR/27257 PR/25551 PR/22182 PR/31095 PR/30504 PR/36816 PR/37860 PR/39308 Also PR/18614 should no longer die, but a little more work needs to be done on the merging for very large files.
2009-08-18The code that attempted to sort large files by sorting each chunk by thedsl
first key byte and writing to a temp file, then sorting the records from each temp file that had the same first key byte (and repeating for upto 4 key bytes) was a nice idea, but completely doomed to failure. Eg PR/9308 where a 70MB file has all but one record the same and short keys. Not only does the code not work, it is rather guaranteed to be slow. Instead always use a merge sort for fully sorted chunk of records (each temporary file contains one lot of sorted records). The -H option already did this, so just rip out all the code and variables that can't be used when -H was specified. Further cleanup to come ...
2009-08-16Replace all uses of sizeof(TRECHEADER) with REC_DATA_OFFSET - whichdsl
is defined as offsetof(RECHEADER, data). Delete TRECHEADER.
2009-08-15Remove reference to db.h by using separate ptr+len fields for the onlydsl
structure that used it. Pass end of keybuf area, not size to enterkey() - largely to remove a variable who'se use isn't obvious from the name! The structute of this code sucks.
2009-08-15linebuf and linebuf_size are only used inside seq() - which also notdsl
only has its own static variable, but will also extend the buffer. Remove linebuf/size and change seq() to use a private, locally managed buffer.
2009-08-15Remove the unused 'DBT *key' parameter from seq().dsl
2009-08-15In makeline() change 'pos' from 'char *' to 'u_char *' and remove alldsl
the casts associated with its use. None of the uses can possibly care about the signedness of the pointer.
2009-08-15Ansify.dsl
I'm looking at fixing the 'sort -n' fubars, but this code is an inpeneterable mess - which needs some fixing first!
2009-04-13Fix WARNS=4 issues (-Wcast-qual -Wsign-compare)lukem
2008-04-28Remove clause 3 and 4 from TNF licensesmartin
2006-05-11char -> u_char in a couple of places to match other variables.mrg
2005-06-07Initialize a local variable to appease -Wuninitialized.he
Marked with XXXGCC for pmppc (found while compiling for it). Reviewed by lukem.
2004-02-15fix some cases of use of unitialized variablesjdolecek
2003-10-18KNF (mostly whitespace)itojun
2003-10-16safer use of reallocitojun
2003-08-07add TNF copyrightjdolecek
2003-08-07Move UCB-licensed code from 4-clause to 3-clause licence.agc
Patches provided by Joel Baker in PR 22365, verified by myself.
2002-12-24max_o in struct tempfile needs to be off_tjdolecek
use fseeko() rather than fseek() when changing file offset using max_o
2001-05-15Make compilable with -Wshadowjdolecek
2001-02-19Pull up various cosmetic (mostly whitespace) changes from OpenBSD.jdolecek
This is primarily to ease syncing the two versions.
2001-02-19resurrect old ftmp() - it supports alternative directory for temporaryjdolecek
file, which is needed for -T support
2001-01-18makeline(): make the overflow handling code safe vs. buffer realloc, addjdolecek
a comment explaining what we do here
2001-01-13makeline(): put back the memmove(3) removed in rev 1.5 in belief it's beenjdolecek
redundant. "Oops" This fixes bug reported to me by Simon Burge.
2001-01-13fix few confusing indentation. XXX still brokenitojun
2001-01-13one more warning to killjdolecek
2001-01-13Since SUS explicitly specifies sort(1) should append a recordjdolecek
delimiter to file if it doesn't end with one, don't warn when this happens.
2001-01-12remove #if 0 partjdolecek
2001-01-11general cleanup of file list passing:jdolecek
* get rid of union f_handle, replace by passing explicit int parameter and (new) struct filelist * add new typedefs gen_func_t and put_func_t and use where appropriate
2001-01-08make ftmp() wrapper aroung tmpfile(), there is no need to reimplement itjdolecek
move ftmp() from tmp.c to files.c g/c no longer needed stuff
2000-10-17fix bugs caused by implicit assumption that 'length' andjdolecek
'offset' members of struct recheader/trecheader are shorts - they are size_t now this makes sort pass all tests in TEST/stests again after my last change other misc cosmetic changes
2000-10-16enlarge line buffer as necessary, so that it's possiblejdolecek
to process lines longer than 65522 characters constify, rename MAXLLEN to DEFLLEN
2000-10-15don't use register declarationsjdolecek
2000-10-07Two classes of changes from the initial OpenBSD commit of this sort(1):bjh21
FILE * variables are called "fp" rather than "fd". Better (safer) temporary-file handling.
2000-10-07Hit sort(1) with a hammer till it compiles.bjh21
Also add RCSIDs.
2000-10-074.4BSD-Lite2 contrib/sortbjh21