Sorry, that my question about memory lead to confusion. I just asked, to
get a bit of an overview, what address range to expect.  (e.g. address
0xa800000033xxxxxx is between 304 and 320MB, the size of the secondary
cache is also interesting, since it determines, which bits from the
physical address make an index into the cache)

All this highmem stuff only concerns 32bit kernels, which we cannot use
on IP28 anyway (one reason is, that here RAM starts at address 512MB).
You should be able to put in up to 1GiB of RAM without problems.

In the attachements you can see, what i could find out about both Oops'es.
(Fortunately the older kernel and System.map matched the latter ones over
a wide range)

About looking up interesting addresses in System.map:
ksymoops often gives us a reference for the trapped instruction relative
only to a local label, e.g.
  >>PC;  a8000000201b51a0 <$L142+8/18>   <=====
since $L142 is a local label (mostly there are several of the same name
in System.map) we look up the function with the next lower address, here
we find
  a8000000201b5100 T vsnprintf
  ...
  a8000000201b5198 t $L142
>> >>PC;  a8000000201b51a0 == $L142+8 == vsnprintf+0xa0
  ...
  a8000000201b57a0 t $L264
  a8000000201b57e8 T snprintf
(of course the trapped address itself usually is not in System.map, and
a 32bit bus error address must be converted to a kernel address first)


Also attached is a more verbose version of ip22-berr.c for the next bus
error.


with kind regards

pf




This case is strange, the facts don't fit together very well, the offending
address is not related to the trapped instruction.
Tracing back, up to do_pipe(), we have the following situation:


  int do_pipe(int *fd)
  {
  /* $29, sp : a800000030317de0 */
    struct qstr this; /* &this = &.name = sp, &.len = sp+8, &.hash = sp+12 */
    char name[32];    /* &name = sp+16 */
    struct dentry *dentry;
    struct inode * inode;
    struct file *f1, *f2;
    int error;
    int i,j;

    ...
    error = -ENOMEM;
    sprintf(name, "[%lu]", inode->i_ino);
    this.name = name;
    this.len = strlen(name);
    this.hash = inode->i_ino; /* will go */
    dentry = d_alloc(pipe_mnt->mnt_sb->s_root, &this);
    ...
  }
  struct dentry * d_alloc(struct dentry * parent, const struct qstr *name)
  {
  /* $29, sp : a800000030317db0 */
    char * str;
    struct dentry *dentry;

    dentry = kmem_cache_alloc(dentry_cache, GFP_KERNEL);
    if (!dentry)
      return NULL;

    if (name->len > DNAME_INLINE_LEN-1) {
      str = kmalloc(NAME_ALLOC_LEN(name->len), GFP_KERNEL);
      if (!str) {
        kmem_cache_free(dentry_cache, dentry);
        return NULL;
      }
    } else
      str = dentry->d_iname;

    memcpy(str, name->name, name->len);
    ...
  }

This efectively results in a call:
 memcpy(dentry->d_iname, name->name, name->len);
with numerical addresses:

  memcpy( a80000002e02cea8, a800000030317df0, 8 );

  vmlinux:     file format elf64-tradbigmips

  Disassembly of section .text:

  a8000000201b34c0 <memcpy>:
  a8000000201b34c0:	0080102d 	move	v0,a0
  a8000000201b34c4 <__copy_user>:
  ...
  a8000000201b35ac <less_than_4units>:
  a8000000201b35ac:	13060040 	beq	t8,a2,a8000000201b36b0 <copy_bytes>
  a8000000201b35b0:	00000000 	nop
  a8000000201b35b4:	dca80000 	ld	a4,0(a1)
  a8000000201b35b8:	64a50008 	daddiu	a1,a1,8
  a8000000201b35bc:	64c6fff8 	daddiu	a2,a2,-8
  a8000000201b35c0 <epc == less_than_4units+0x14/48 == memcpy+0x100>:
  a8000000201b35c0:	fc880000 	sd	a4,0(a0)
  ...
  a8000000201b3720 <done>:
  a8000000201b3720:	03e00008 	jr	ra
  a8000000201b3724:	00000000 	nop

  GIO error 0x410:<TIME > @ 0xb0317de0	# PA(a800000030317de0 | 80000000)
  $1, at  : ffffffff9004cce0
  $2, v0  : a80000002e02cea8 0000000000000000
  $4, a0  : a80000002e02cea8 a800000030317df8 0000000000000000 ffffffff9004cce1
  $8      : 5b3130363338335d 0000000000000000 0000000000000000 a800000030317df7
  $12, t0 : ffffffffffffffff a8000000b0317dee 0000000000000000 a8000000201e60c0
  $16, s0 : a80000002e02ce00 a800000033ffb440 a800000030317de0 a80000002e02cea8
  $20, s4 : fffffffffffffff4 0000000000000006 0000000000000005 a800000030317e60
  $24, t8 : 0000000000000000 0000000000000000
  $28, gp : a800000030314000
  $29, sp : a800000030317db0
  $30     : 0000000000000001
  $31, ra : a8000000200707fc


The offending address could be interpreted as the physical (bus) address
of a (unmapped) kernel address a800000030317de0 corrupted by setting bit
31 to 1 (or PA taken first, then corrupted, or...).
Btw. it is the first variable's address in do_pipe().

The trapped instruction "sd a4,0(a0)" stores to a80000002e02cea8, which
is unrelated to the offending address. A more likely candidate (assuming
corruption of a valid address) would be the "ld a4,0(a1)" some lines
before, which accesses a800000030317df8.

The disassembly and source of the functions in the call trace show no code,
that would do such corruption (e.g. by address conversions with 32bit KSEG),
and the registers a0 and a1, as arguments to memcpy, show correct values.

Like in the vsnprintf() case we find the offending address + 0xe in one
register, this time $13 alias t1. But here the register isn't even used
in any function from the call trace.
But if some code outside this range should have set the register (and
used it) and managed somehow to create a cache tag with the invalid
address, the bus error should have occurred as soon as do_pipe() writes
to the "this" variable, consequently forcing the processor to write back
and evict the cache block for the invalid tag to make room for the
variable's cache block (the position in the cache is defined by some middle
bits of the physical address, which here are the same for correct and
invalid address).

As a summary: this case is quite obscure...
(especially since EPC doesn't seem to point to the offending instruction)





GIO error 0x401:<TIME > @ 0xa38cfff0

Here the offending address is PA(0xa8000000a38cfffe) aligned to quadword,
and 0xa8000000a38cfffe is a pointer ("end") to the last byte of a print
buffer ("buf"), see register $17 (alias s1) below.

$1, at  : 00000000201c13e0
$2, v0  : a8000000201c1400 000000000000003a
$4, a0  : 000000000000003a 000000007fffffff a8000000201c13ff a800000028fc3cd0
$8      : 0000000003631000 0000000000000000 00000000006be000 0000000001b1f000
$12, t0 : 0000000000000000 0000000000000000 0000000000000000 0000000000000003
$16, s0 : a8000000238d0020 a8000000a38cfffe 0000000000001000 000000002aac3000
$20, s4 : a800000028fc3cd0 a800000028fc3e00 a8000000238d0000 000000007fffffff
$24, t8 : 0000000000000001 000000002ab7aafc
$28, gp : a800000028fc0000
$29, sp : a800000028fc3c50
$30     : a800000028fc3e08
$31, ra : a8000000201b5838


Call trace:

int sprintf(char * buf, const char *fmt, ...)
{
  ...
  i=vsprintf(buf,fmt,args);
  ...
}
int vsprintf(char *buf, const char *fmt, va_list args)
{
  return vsnprintf(buf, (~0U)>>1, fmt, args);
}

int vsnprintf(char *buf, size_t size, const char *fmt, va_list args)
buf  == a8000000238d0000 (s6)
size == 000000007fffffff (a1,s7)
{
    int len;
    unsigned long long num;
    int i, base;
    char *str, *end, c;
    const char *s;
    ...
    str = buf;
    end = buf + size - 1; /* a8000000a38cfffe = a8000000238d0000+7fffffff-1 */
    ...
    for (; *fmt /* epc points here, but ... */; ++fmt) {      
    ...
    }
    if (str <= end)
      *str = '\0';
    else if (size > 0)
      /* ... it's only here, where the exception makes sense */
      *end = '\0';
    ...
}

vmlinux:     file format elf64-tradbigmips

Disassembly of section .text:

a8000000201b5100 <vsnprintf>:
a8000000201b5100:	67bdffa0 	daddiu	sp,sp,-96
a8000000201b5104:	0005103c 	dsll32	v0,a1,0x0
a8000000201b5108:	0002103f 	dsra32	v0,v0,0x0
a8000000201b510c:	ffb70048 	sd	s7,72(sp)
a8000000201b5110:	ffb60040 	sd	s6,64(sp)
a8000000201b5114:	ffb40030 	sd	s4,48(sp)
a8000000201b5118:	ffbf0050 	sd	ra,80(sp)
a8000000201b511c:	ffb50038 	sd	s5,56(sp)
a8000000201b5120:	ffb30028 	sd	s3,40(sp)
a8000000201b5124:	ffb20020 	sd	s2,32(sp)
a8000000201b5128:	ffb10018 	sd	s1,24(sp)
a8000000201b512c:	ffb00010 	sd	s0,16(sp)
a8000000201b5130:	00a0b82d 	move	s7,a1
a8000000201b5134:	0080b02d 	move	s6,a0
a8000000201b5138:	ffa60000 	sd	a2,0(sp)
a8000000201b513c:	0440018e 	bltz	v0,a8000000201b5778 <$L255>
a8000000201b5140:	00e0a02d 	move	s4,a3
a8000000201b5144:	0085182d 	daddu	v1,a0,a1
a8000000201b5148:	6471ffff 	daddiu	s1,v1,-1
a8000000201b514c:	6482ffff 	daddiu	v0,a0,-1
a8000000201b5150:	0222102b 	sltu	v0,s1,v0
a8000000201b5154:	10400003 	beqz	v0,a8000000201b5164 <$L139>
a8000000201b5158:	0080802d 	move	s0,a0
a8000000201b515c:	2411ffff 	li	s1,-1
a8000000201b5160:	0004b82f 	dnegu	s7,a0
a8000000201b5164 <$L139>:
a8000000201b5164:	80c20000 	lb	v0,0(a2)
a8000000201b5168:	10400011 	beqz	v0,a8000000201b51b0 <$L239>
a8000000201b516c:	00c0202d 	move	a0,a2
a8000000201b5170 <$L234>:
a8000000201b5170:	80830000 	lb	v1,0(a0)
a8000000201b5174:	24020025 	li	v0,37
a8000000201b5178:	10620023 	beq	v1,v0,a8000000201b5208 <$L144>
a8000000201b517c:	90840000 	lbu	a0,0(a0)
a8000000201b5180:	0230102b 	sltu	v0,s1,s0
a8000000201b5184:	54400004 	bnezl	v0,a8000000201b5198 <$L142>
a8000000201b5188:	66100001 	daddiu	s0,s0,1
a8000000201b518c:	a2040000 	sb	a0,0(s0)
a8000000201b5190 <$L254>:
a8000000201b5190:	dfa60000 	ld	a2,0(sp)
a8000000201b5194:	66100001 	daddiu	s0,s0,1
a8000000201b5198 <$L142>:
a8000000201b5198:	64c20001 	daddiu	v0,a2,1
a8000000201b519c:	ffa20000 	sd	v0,0(sp)

# Here ("*fmt;") the exception is trapped, ...
a8000000201b51a0 <epc == $L142+8/18 == vsnprintf+0xa0>:
a8000000201b51a0:	80430000 	lb	v1,0(v0)
... 25 lines skipped
a8000000201b51f8 <$L235>:
a8000000201b51f8:	56e0fff1 	bnezl	s7,a8000000201b51c0 <$L236>
# ...but here ("*end(== a8000000a38cfffe) = 0;") it would make sense
a8000000201b51fc:	a2200000 	sb	zero,0(s1)
a8000000201b5200:	0806d471 	j	a8000000201b51c4 <$L265>
... 436 lines skipped


So doing "*end = 0;" could be a cause for the bus error. This statement
will never be reached in regular execution due to the high value of end.
But it is possible, that this instructions will be started by wrong branch
prediction and speculative execution, which would cause a bus error even
when the instruction will be cancelled later.

The difficulties with this assumption are
- the ExceptionProgramCounter points to a different instruction, not
  related in any way to the offending address.
- it is not very likely, that it happens only after a hour, vsnprintf
  is probably called earlier and often ?

(I had experienced speculative loop execution beyond available RAM in a
certain configuration, but then EPC precisely pointed to the offending
instruction)

Should the assumption turn out true, the problem could be avoided for
this very case by calling vsnprintf() with a reasonable "size" parameter,
in general and globally we would need compiler support.