I-Appliance BBS
The Official Source for Internet Appliance Upgrades and Mods

Click Here!
BBS Main List | Sign In | Sign Up | Search | Help | Linux-Hacker.netReply to Thread | Printer |

Home / DigitalVideoRecorders / Tivo Hacks
Problems with ExtractStream.

New MessageProblems with ExtractStream. (modified 0 times) dralston
Profile
Thank you for releasing your extraction software. The attitude of the AVS forum was really pissing me off. Hopefully this forum will have much more discussion of hacking.

Anyway I can't seem to get ExtractStream to extract anything. Here's what I get:

Attempting to locate tyStream with fsid 404361...
...tyStream located, sector map follows:
Start Length
02c13c00 00008000
02c1bc00 00004000
Block: 2c13c00 Count: 32768 Total blocks 2
/dev/hda10 has 0x00100000 sectors
/dev/hda11 has 0x02811f00 sectors
/dev/hdb2 has 0x00002000 sectors
/dev/hdb3 has 0x04c53c00 sectors
Sequence number bogus 269ff4!=ffbd1f at record 7 of 18
Bad chunk, skippingf00
Sequence number bogus 269ff4!=205640 at record 0 of 75

and a bunch more of the "Sequence number bogus" errors. I can't
imagine what I'm doing differently than anyone else.

Also my roommate can't seem to compile the TiVo toolchain. Anyone have any tips on this? Has anyone compiled it with the version of gcc that comes with the latest RedHat distro?

06-09-2001 00:23:37

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
This is just an over-zealous printf... similar to PlayStream's "checksum error". There are still some concerns that the sequence recycling indication is correct, and that audio packets remain sync'd, so these printf's remain.
06-09-2001 13:13:47

New MessageCompile errors (modified 0 times) topset
Profile
The AVS TiVo Disney Underground has plenty of help and links for compiling the toolchain. Search there. There are some scripts to automate the process.
06-09-2001 13:16:12

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
I'm the roommate with the problems building the toolchain, but I finally got it working. What I ended up doing was using Tridge's binaries, except that I built libc myself from TiVo's sources. (Without replacing libc, I was getting linking errors on anything I tried to compile. I guessed that the missing symbols belonged to libc, and it seems I guessed right.)

Anyway, as I posted on the Yahoo mailing list, when I took out the code that checked the sequence numbers, I got a valid audio stream, but a seemingly invalid video stream. The sequence error definitely causes the program to not output part of the stream, and we were getting so many of these errors that we were left with virtually nothing. (A two minute clip made a two second audio stream!) I'm still wondering why the video stream is invalid.

-Mark

06-09-2001 18:54:07

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
>The sequence error definitely causes the program to not output part of the stream

The v1.3 PlayStream "checksums" were replaced in 2.0.1 with Sequences.

Look in the ExtractStream code for:

if (!PredictSequence ||
TempSequence == 0x205040 ||
TempSequence == 0x205048) // start/restart
PredictSequence = TempSequence;

So far, we've found those two numbers as the "key" to restarting the sequence.

You're only showing two out-of-sequence messages... look at more... see if you can find another sequence restarter, like maybe the "205640" in your printout.

06-09-2001 19:25:07

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
The complete output follows. It's interesting that the expected sequence number is always the same. Anyway, I'm most interested right now in why I can't get a valid video stream. (Though strangely when I have ExtractStream send the streams to /dev/mpeg0a and /dev/mpeg0v, the TiVo plays the video but not the audio!)

bash-2.02# ExtractStream 404361
Attempting to locate tyStream with fsid 404361...
...tyStream located, sector map follows:
Start Length
02c13c00 00008000
02c1bc00 00004000
Block: 2c13c00 Count: 32768 Total blocks 2
/dev/hda10 has 0x00100000 sectors
/dev/hda11 has 0x02811f00 sectors
/dev/hdb2 has 0x00002000 sectors
/dev/hdb3 has 0x04c53c00 sectors
Sequence number bogus 269ff4!=ffbd1f at record 7 of 18
Bad chunk, skippingf00
Sequence number bogus 269ff4!=205640 at record 0 of 75
Sequence number bogus 269ff4!=222bd0 at record 0 of 114
Sequence number bogus 269ff4!=23df4c at record 0 of 74
Sequence number bogus 269ff4!=25aefc at record 0 of 115
Sequence number bogus 269ff4!=275b78 at record 0 of 95
Sequence number bogus 269ff4!=291a94 at record 7 of 96
audio failure 0x814:0:0x288 0xf9:0xf3:0xe5
Sequence number bogus 269ff4!=2ad480 at record 0 of 111
Sequence number bogus 269ff4!=2c8b94 at record 0 of 109
Sequence number bogus 269ff4!=2176e4 at record 3 of 88
audio failure 0x794:0:0x304 0xc8:0x1d:0x82
Sequence number bogus 269ff4!=2337b8 at record 0 of 104
Sequence number bogus 269ff4!=24ec5c at record 0 of 92
Sequence number bogus 269ff4!=26a5b8 at record 0 of 91
Sequence number bogus 269ff4!=286ccc at record 0 of 89
Sequence number bogus 269ff4!=2a2648 at record 7 of 93
audio failure 0x7e4:0:0x64 0xdc:0x88:0xe1
Sequence number bogus 269ff4!=2be964 at record 0 of 114
Sequence number bogus 269ff4!=20d4b4 at record 0 of 116
Sequence number bogus 269ff4!=227db0 at record 0 of 81
Sequence number bogus 269ff4!=244ce0 at record 0 of 118
Sequence number bogus 269ff4!=25f994 at record 0 of 100
Sequence number bogus 269ff4!=27b1f4 at record 0 of 97
Sequence number bogus 269ff4!=2974c0 at record 0 of 98
Sequence number bogus 269ff4!=2b3480 at record 0 of 92
Sequence number bogus 269ff4!=2cf7c0 at record 7 of 100
Sequence number bogus 269ff4!=21e0b4 at record 0 of 109
Sequence number bogus 269ff4!=238e08 at record 0 of 84
Sequence number bogus 269ff4!=255c00 at record 0 of 73
Sequence number bogus 269ff4!=2728b4 at record 0 of 125
Sequence number bogus 269ff4!=28cdb4 at record 0 of 108
Sequence number bogus 269ff4!=2a7ed0 at record 5 of 94
audio failure 0x7f4:0:0xac 0x97:0xf5:0xa1
Sequence number bogus 269ff4!=2c44f8 at record 0 of 89
Sequence number bogus 269ff4!=213970 at record 0 of 100
Sequence number bogus 269ff4!=22f230 at record 0 of 101
Sequence number bogus 269ff4!=24a9ec at record 0 of 96
Sequence number bogus 269ff4!=265f94 at record 0 of 99
Sequence number bogus 269ff4!=281ed0 at record 0 of 101
Sequence number bogus 269ff4!=29da84 at record 0 of 107
Sequence number bogus 269ff4!=2b8f5c at record 0 of 83
Sequence number bogus 269ff4!=208f98 at record 13 of 107
audio failure 0x8c4:0:0x190 0xa3:0x79:0xf3
Sequence number bogus 269ff4!=2249ac at record 0 of 91
Sequence number bogus 269ff4!=23ffb8 at record 0 of 110
Sequence number bogus 269ff4!=25b908 at record 3 of 96
audio failure 0x814:0:0x234 0x98:0x9a:0x2d
Sequence number bogus 269ff4!=277a14 at record 0 of 100
Sequence number bogus 269ff4!=292ef0 at record 7 of 104
Sequence number bogus 269ff4!=2aeae4 at record 0 of 95
Sequence number bogus 269ff4!=2cb128 at record 5 of 100
Sequence number bogus 269ff4!=2193b0 at record 0 of 97
Sequence number bogus 269ff4!=235680 at record 0 of 115
Sequence number bogus 269ff4!=250014 at record 0 of 96
Sequence number bogus 269ff4!=26c358 at record 0 of 79
Sequence number bogus 269ff4!=288f00 at record 5 of 139
audio failure 0xba4:0:0x9c 0x8a:0x9c:0x45
Sequence number bogus 269ff4!=2a248c at record 0 of 78
Sequence number bogus 269ff4!=2bf3e8 at record 0 of 99
Sequence number bogus 269ff4!=20dcf0 at record 0 of 93
Sequence number bogus 269ff4!=22a0ac at record 6 of 94
audio failure 0x7f4:0:0xa8 0xc5:0x2f:0x4e
Sequence number bogus 269ff4!=24635c at record 0 of 99
Sequence number bogus 269ff4!=261bc0 at record 0 of 107
Sequence number bogus 269ff4!=27c944 at record 0 of 85
Sequence number bogus 269ff4!=299b04 at record 5 of 104
Sequence number bogus 269ff4!=2b4c38 at record 0 of 98
Sequence number bogus 269ff4!=2d0130 at record 4 of 82
Sequence number bogus 269ff4!=21fcf0 at record 0 of 110
Sequence number bogus 269ff4!=23b128 at record 0 of 107
Sequence number bogus 269ff4!=255b4c at record 0 of 76
Sequence number bogus 269ff4!=272e38 at record 0 of 100
Sequence number bogus 269ff4!=28e310 at record 0 of 91
Sequence number bogus 269ff4!=2ab108 at record 0 of 102
Sequence number bogus 269ff4!=2c5ef4 at record 0 of 112
Sequence number bogus 269ff4!=214704 at record 0 of 107
Sequence number bogus 269ff4!=23058c at record 0 of 97
Sequence number bogus 269ff4!=24c190 at record 0 of 94
Sequence number bogus 269ff4!=267768 at record 0 of 107
Sequence number bogus 269ff4!=282f30 at record 0 of 102
Sequence number bogus 269ff4!=29f198 at record 0 of 92
Sequence number bogus 269ff4!=2bb604 at record 5 of 93
audio failure 0x7e4:0:0xc8 0x13:0x7e:0xfb
Sequence number bogus 269ff4!=20a954 at record 0 of 95
Sequence number bogus 269ff4!=226270 at record 0 of 103
Sequence number bogus 269ff4!=241714 at record 0 of 98
Sequence number bogus 269ff4!=25cd24 at record 9 of 93
audio failure 0x7e4:0:0x168 0x3a:0xce:0xb3
Sequence number bogus 269ff4!=2792a0 at record 0 of 94
Sequence number bogus 269ff4!=29486c at record 0 of 106
Sequence number bogus 269ff4!=2affdc at record 0 of 98
Sequence number bogus 269ff4!=2cc2f8 at record 0 of 102
Sequence number bogus 269ff4!=21ac38 at record 0 of 90
Sequence number bogus 269ff4!=2372fc at record 0 of 103
Sequence number bogus 269ff4!=25281c at record 0 of 101
Sequence number bogus 269ff4!=26dfe4 at record 0 of 88
Sequence number bogus 269ff4!=28a3d8 at record 0 of 112
Sequence number bogus 269ff4!=2a4d98 at record 0 of 102
Sequence number bogus 269ff4!=2c0ff4 at record 0 of 93
Sequence number bogus 269ff4!=21040c at record 0 of 95
Sequence number bogus 269ff4!=22c768 at record 0 of 103
Sequence number bogus 269ff4!=247550 at record 0 of 110
Sequence number bogus 269ff4!=262978 at record 0 of 84
Sequence number bogus 269ff4!=27e66c at record 0 of 79
Sequence number bogus 269ff4!=29b1dc at record 0 of 96
Sequence number bogus 269ff4!=2b6ae4 at record 0 of 98
Sequence number bogus 269ff4!=205e74 at record 1 of 104
Sequence number bogus 269ff4!=221670 at record 0 of 107
Sequence number bogus 269ff4!=23ca6c at record 0 of 89
Sequence number bogus 269ff4!=258eb4 at record 0 of 107
Sequence number bogus 269ff4!=274610 at record 0 of 90
Sequence number bogus 269ff4!=2909c8 at record 0 of 105
Sequence number bogus 269ff4!=2ab77c at record 0 of 101
Sequence number bogus 269ff4!=2c76a4 at record 0 of 103
Sequence number bogus 269ff4!=216330 at record 0 of 89
Sequence number bogus 269ff4!=232dd4 at record 0 of 112
Sequence number bogus 269ff4!=24da84 at record 0 of 93
Sequence number bogus 269ff4!=269aa8 at record 0 of 88
Sequence number bogus 269ff4!=285e90 at record 0 of 129
Sequence number bogus 269ff4!=2a0698 at record 0 of 89
Sequence number bogus 269ff4!=2bc028 at record 0 of 92
Sequence number bogus 269ff4!=20b7c8 at record 0 of 96
Sequence number bogus 269ff4!=227ab8 at record 0 of 94
Sequence number bogus 269ff4!=243a00 at record 11 of 110
audio failure 0x8f4:0:0x320 0x81:0x27:0xd6
Sequence number bogus 269ff4!=25eb84 at record 0 of 67
Sequence number bogus 269ff4!=27c5e8 at record 0 of 130
Sequence number bogus 269ff4!=296714 at record 0 of 100
Sequence number bogus 269ff4!=2b263c at record 0 of 98
Sequence number bogus 269ff4!=2ce5ec at record 0 of 111
Sequence number bogus 269ff4!=21c6a4 at record 0 of 88
Sequence number bogus 269ff4!=23862c at record 5 of 96
audio failure 0x814:0:0x26c 0x2d:0x7f:0x75
Sequence number bogus 269ff4!=254a78 at record 0 of 97
Sequence number bogus 269ff4!=270310 at record 0 of 101
Sequence number bogus 269ff4!=28bbc0 at record 0 of 101
Sequence number bogus 269ff4!=2a7acc at record 0 of 96
Block: 2c1bc00 Count: 16384 Total blocks 2
Sequence number bogus 269ff4!=2a7acc at record 0 of 96
Sequence number bogus 269ff4!=2c36ec at record 0 of 95
Sequence number bogus 269ff4!=212ae0 at record 0 of 109
Sequence number bogus 269ff4!=22d840 at record 0 of 88
Sequence number bogus 269ff4!=24a2f4 at record 0 of 118
Sequence number bogus 269ff4!=264bdc at record 0 of 94
Sequence number bogus 269ff4!=280f58 at record 0 of 97
Sequence number bogus 269ff4!=29cbbc at record 0 of 88
Sequence number bogus 269ff4!=2b9634 at record 7 of 132
Sequence number bogus 269ff4!=207214 at record 0 of 81
Sequence number bogus 269ff4!=2239e4 at record 0 of 107
Sequence number bogus 269ff4!=23eae0 at record 0 of 88
Sequence number bogus 269ff4!=25aebc at record 0 of 98
Sequence number bogus 269ff4!=276aa0 at record 0 of 114
Sequence number bogus 269ff4!=291e74 at record 0 of 85
Sequence number bogus 269ff4!=2ad840 at record 0 of 98
Sequence number bogus 269ff4!=2c9b0c at record 0 of 95
Sequence number bogus 269ff4!=21888c at record 4 of 97
audio failure 0x824:0:0xd4 0x4b:0x52:0xe4
Sequence number bogus 269ff4!=234788 at record 0 of 97
Sequence number bogus 269ff4!=250020 at record 0 of 108
Sequence number bogus 269ff4!=26ad84 at record 0 of 104
Sequence number bogus 269ff4!=286900 at record 0 of 84
Sequence number bogus 269ff4!=2a2d24 at record 0 of 103
Sequence number bogus 269ff4!=2bdeac at record 3 of 92
audio failure 0x844:0:0x38 0xb2:0x6b:0x58
Sequence number bogus 269ff4!=20dc5c at record 0 of 95
Sequence number bogus 269ff4!=229214 at record 0 of 99
Sequence number bogus 269ff4!=245520 at record 0 of 98
Sequence number bogus 269ff4!=260d98 at record 0 of 93
Sequence number bogus 269ff4!=27c6e4 at record 0 of 102
Sequence number bogus 269ff4!=2985dc at record 0 of 112
Sequence number bogus 269ff4!=2b3318 at record 0 of 88
Sequence number bogus 269ff4!=2d0138 at record 0 of 113
Sequence number bogus 269ff4!=21e250 at record 0 of 89
Sequence number bogus 269ff4!=239f44 at record 0 of 96
Sequence number bogus 269ff4!=2557f8 at record 0 of 105
Sequence number bogus 269ff4!=27132c at record 3 of 98
audio failure 0x7d4:0:0x44 0x65:0xf9:0x58
Sequence number bogus 269ff4!=28d304 at record 0 of 94
Sequence number bogus 269ff4!=2a9614 at record 0 of 109
Sequence number bogus 269ff4!=2c4db8 at record 0 of 101
Sequence number bogus 269ff4!=2136fc at record 0 of 94
Sequence number bogus 269ff4!=22fa00 at record 0 of 101
Sequence number bogus 269ff4!=24b2b0 at record 0 of 99
Sequence number bogus 269ff4!=2671ec at record 0 of 110
Sequence number bogus 269ff4!=282980 at record 0 of 99
Sequence number bogus 269ff4!=29ec28 at record 0 of 114
Sequence number bogus 269ff4!=2b92c4 at record 0 of 86
Sequence number bogus 269ff4!=208c94 at record 9 of 102
audio failure 0x874:0:0x224 0x47:0xd3:0x65
Sequence number bogus 269ff4!=22466c at record 0 of 81
Sequence number bogus 269ff4!=2411c4 at record 0 of 112
Sequence number bogus 269ff4!=25c928 at record 0 of 111
Sequence number bogus 269ff4!=277904 at record 2 of 105
audio failure 0x8a4:0:0x300 0x57:0x32:0x3d
Sequence number bogus 269ff4!=292e60 at record 0 of 96
Sequence number bogus 269ff4!=2aedbc at record 0 of 86
Sequence number bogus 269ff4!=2cb1d4 at record 0 of 95
Sequence number bogus 269ff4!=21a5c8 at record 0 of 117
Sequence number bogus 269ff4!=2347e8 at record 0 of 88
Sequence number bogus 269ff4!=251608 at record 0 of 107
Sequence number bogus 269ff4!=26cc14 at record 7 of 103
audio failure 0x8f4:0:0x194 0x25:0x35:0x53
Sequence number bogus 269ff4!=2885e8 at record 11 of 100
Sequence number bogus 269ff4!=2a3b48 at record 0 of 94
Sequence number bogus 269ff4!=2c0594 at record 0 of 80
Sequence number bogus 269ff4!=210184 at record 0 of 126
Sequence number bogus 269ff4!=22a9c4 at record 0 of 100
Sequence number bogus 269ff4!=246284 at record 0 of 95
Sequence number bogus 269ff4!=2621fc at record 0 of 98
Flushing buffers

06-09-2001 19:38:06

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
Those audio failures are pretty bad too.

Can you make a 1MB raw (use the -s option) snipet, make a freedrive.com account and put the snipet on that account (or similar) and post the username/password so we can look at it?

What kind of TiVo do you have? What version is it running? This has only been tested on 2.0.1.

06-09-2001 20:36:09

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
Okay, you can get it from here:

http://www.baysinger.org/~mark/404361.raw

(I just did ctrl-c after a couple seconds, and ended up with a 1MB file.)

Is that suitable? Strange that the the errors are in the audio, since that is the stream that mplex and Windows Media recognized, and I can't hear any errors in it when I play it with Windows Media.

-Mark

06-09-2001 21:22:52

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
Oh, almost forgot:

Philips, stand-alone, 2.0.1-001-000

06-09-2001 21:26:54

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
The Guru who works on these things has a night job... he wont be able to look at it until morning...

>It's interesting that the expected sequence number is always the same.

That's because the code can't recover from a bad sequence.

I have a hunch: the "269ff4!=ffbd1f" was a bad sequence, and the next "269ff4!=205640" was a start of a good sequence. Could you add:

TempSequence == 0x205640

To the predicate:

if (!PredictSequence ||
TempSequence == 0x205040 ||
TempSequence == 0x205048) // start/restart

Sorry for all the hassle. This definitely comes from having to keep this code such a secret, and with all the folks that new the changes keeping their lips sealed.

06-09-2001 21:49:24

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
Thanks, that cleared up most of the error messages. There were still two sequence errors, and the video stream was not valid, as before.

bash-2.02# ./a.out 404361
Attempting to locate tyStream with fsid 404361...
...tyStream located, sector map follows:
Start Length
02c13c00 00008000
02c1bc00 00004000
Block: 2c13c00 Count: 32768 Total blocks 2
/dev/hda10 has 0x00100000 sectors
/dev/hda11 has 0x02811f00 sectors
/dev/hdb2 has 0x00002000 sectors
/dev/hdb3 has 0x04c53c00 sectors
Sequence number bogus 269ff4!=ffbd1f at record 7 of 18
Bad chunk, skippingf00
Block: 2c1bc00 Count: 16384 Total blocks 2
Sequence number bogus 2c36ec!=2a7acc at record 0 of 96
Flushing buffersc1fc00

06-09-2001 22:32:12

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
What do you mean by "the video was not valid"?

Some of these messages are to be expected.

If it didn't play through the mpeg device... that may not be a problem... the instructions say that hasn't been tested for some time... the PES headers are stripped.

Have you tried playing the file?

06-09-2001 22:37:42

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
Windows Media Player doesn't recognize it, and mplex reports:

INFO: mplex version 1.4.1 (7.12.2000)
INFO: File blah.m2a is a 11172-3 Audio stream.
INFO: File blah.m2v is not a MPEG-1/2 Video stream.

The video played fine, however, when ExtractStream directed it to /dev/mpeg0v. Are you able to reproduce this with the raw stream I provided?

-Mark

06-09-2001 22:48:26

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
>INFO: File blah.m2v is not a MPEG-1/2 Video stream.

Which version of mplex are you using? What machine were you running it on?

1) try switching the order of the A/V files on the command line.

2) In linux, type: "od -x <filename> | more" and let's see the first 2 or three lines.

> Are you able to reproduce this with the raw stream I provided?

I don't have the tools to do this. The guy that figured out the sequences works at night... he'll give it a go Monday if we haven't figured it out first.

06-09-2001 23:06:00

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
It's mplex 1.4.1, according to the output. I installed it from http://download.sourceforge.net/mjpeg/mjpegtools-1.4.0-2.rh70.i386.rpm. I'm running Redhat 7.1.

When I switch the parameter order:

$ mplex -f 2 -m 2 -o blah.mpg blah.m2v blah.m2a
INFO: mplex version 1.4.1 (7.12.2000)
**ERROR: Files blah.m2v and blah.m2a are not valid MPEG streams.

Here's the first bit of the video stream:

0000000 a566 ca4d b61f 6443 b18a 6e5b 03a0 af9a
0000020 9c99 113b df1d 2c13 5d96 9f30 a665 ecf9
0000040 154b 3814 777b 040a f22f f7f8 b05a a8db
0000060 f2b5 ae44 52b8 63a8 1173 405d 7d16 72dd
0000100 2dcb de55 b62c 292c 2c2a a328 dd9b 799b
0000120 e5cd c86d e2ed e65b 5ba6 f623 6dc9 5e99

06-09-2001 23:25:51

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
>It's mplex 1.4.1, according to the output. I installed it from http://download.sourceforge.net/mjpeg/mjpegtools-1.4.0-2.rh70.i386.rpm. I'm running Redhat 7.1.

I know that sounded lame, but I've got 3 mplex's that report themselves as 1.4.1... you've got the mjpegtools version (which is good).

>0000000 a566 ca4d b61f 6443 b18a 6e5b 03a0 af9a

That's definitely a bad start. It should start with: 00 00 01

The reason I ask, was there has been some confusion as to the start of the recording.

There's a test for the first frame, that, for some reason you didn't hit:

if (i == StartingRecord && foundfirstframe) { // skip incomplete Video blocks
while (buf[p] != 0 || buf[p+1] != 0 || buf[p+2] != 1) {
p--; // I don't know if this is correct
SizeOffset++; // It makes bad data work ok
if (SizeOffset > 100) { // arbitrary bail
fprintf(stderr,"Can't find first frame :(
");
exit(1);
}
}

From that code, you should have hit that error if it didn't find the correct start within 100 bytes.

Can you see any reason why that error wasn't printed?

How big was the resulting video file?

The tyStream blocks say:

02c13c00 00008000
02c1bc00 00004000

So, it should have counted thru ~40k blocks.

Did it?

06-09-2001 23:53:36

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
What if "foundfirstframe" never happens in the block?

The code that detects the first frame is:

if (!foundfirstframe) {
if (buf[p+3] == 0xE0) {
if (buf[p+2] == 0x8C) {// We've just passed the start of a Video header
foundfirstframe = 1;

If that "8c e0" isn't found, then "foundfirstframe" won't get set.

Note that the parse routine makes two "for (i=" passes thru the header, then a third pass goes thru the data. Test for "foundfirstframe" there, and "fprintf(stderr," a message before the "for" loop if it's not set, and immediately return.

Either you've got a different code for start of the first PES header, or the first valid block doesn't contain a header.

If it's the latter, then the "return" will fix it. If it's the former, then we'll see the error message, and need to dig thru the raw dump you've posted to figure out what your first frame header code indicator is.

06-10-2001 00:09:34

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
Worst case... or temporary fix: find the first header by brute force search thru the video packets:

p = num_recs * 16 + 4;
if (!foundfirstframe) for (i=0; i<j; i++) {
if (TyStream.type == VIDEO_ID) {
long k;
for (k=0; k<TyStream.size; k++) (
if (buf[p+k] == 0 &&
buf[p+k+1] != 0 &&
buf[p+k+2] != 1) {
foundfirstframe = 1;
StartingRecord = i;
goto foundfirst;
}
}
}
p +=TyStream.size;
}
foundfirst:

Put this right before the second "for" loop in the "parse_chunk" procedure, i.e. before:

for (i=StartingRecord; i<j; i++) {

This isn't tested, but should work if skipping the chunk altogether didn't work (by returning if firstframe wasn't set). If skipping the chunk worked (it found the right tag in the chunk header on a subsequent chunk), then that "if (!foundfirstframe) return;" would be better placed before the second "for" loop, rather than the third.

I really wish this editor would allow me to indent!

06-10-2001 07:46:59

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
Oops. Those "!=" should be "==", change:

buf[p+k+1] != 0 &&
buf[p+k+2] != 1

To:

buf[p+k+1] == 0 &&
buf[p+k+2] == 1

06-10-2001 07:50:43

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
Indeed, the first frame doesn't seem to appear until the fourth chunk. Now I'm getting a stream that appeases mplex. (Well, sorta. I'm getting a bunch of "Likely buffer under-run" errors, after a few seconds of output, but I think there was something about that in the ExtractStream docs.) Windows Media still doesn't like the video stream ("not enough storage") or the partial multiplexed stream ("format not supported"), but PowerDVD plays both perfectly.

I did notice the last time I listened to the audio stream before I made this latest change, there was a split second of another audio stream at the beginning. This is gone now. I guess that extra stuff must have been in those first three chunks.

Thanks for all your help!

-Mark

06-10-2001 20:11:20

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
I looked at the ExtractStream documentation again, and I guess I was wrong about the buffer under-run being mentioned. But, I tried with the version of mplex for tivo, and it worked fine. The resulting stream played perfectly in PowerDVD. (Windows Media played audio only. It's probably that I just don't have the right codec, oh well.) So, ExtractStream seems to be working pretty well now, which I'm very happy about!

-Mark

06-10-2001 20:40:25

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
> Indeed, the first frame doesn't seem to appear until the fourth chunk.

So the "return if !firstframe" method worked (or was it the "brute force")?

>"Likely buffer under-run" errors, after a few seconds of output, but I think there
was something about that in the ExtractStream docs.

No, that dealt with the partial audio blocks (which IS still a problem).

This looks like a new error -- which program generated it (I can't find "under[-]run" in any of the mplex's or ExtractStream?

>I guess that extra stuff must have been in those first three chunks.

Digging out all the crap in 2.0.1 tyStreams has been the hard part.

> Thanks for all your help!

Thank you! So far, we've tackled 2 more bugs, maybe a third one -- and Nick didn't even have to figure it out!

06-10-2001 20:45:07

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
> So the "return if !firstframe" method worked

Yep.

> This looks like a new error -- which program generated it (I can't
> find "under[-]run" in any of the mplex's or ExtractStream?

Mplex. Here's what it output:

$ mplex -f 2 -m 2 -o blah.mpg blah.m2[av]
INFO: mplex version 1.4.1 (7.12.2000)
INFO: File blah.m2a is a 11172-3 Audio stream.
INFO: File blah.m2v is a MPEG-1/2 Video stream.
INFO: Scanning Video stream for access units information.
INFO: VIDEO STREAM:
INFO: nStream length : 21469990
INFO: Sequence start : 219
INFO: Sequence end : 0
INFO: No. Pictures : 3499
INFO: No. Groups : 219
INFO: No. I Frames : 219 avg. size 21131 bytes
INFO: No. P Frames : 987 avg. size 7604 bytes
INFO: No. B Frames : 2293 avg. size 4071 bytes
INFO: No. D Frames : 0 avg. size 0 bytes
INFO: Horizontal size: 352
INFO: Vertical size : 480
INFO: Aspect ratio : 0.6735
INFO: Picture rate : 29.970 frames/sec
INFO: Bit rate : 1470000 bits/sec
INFO: Computed rate : 1471200 bits/sec
INFO: Peak rate : 2648400 bits/sec
INFO: Vbv buffer size: 229376 bytes
INFO: CSPF : 0
INFO: Scanning Audio stream for access units information.
INFO: Done, stream bit offset 22381056.
INFO: AUDIO STREAM:
INFO: Audio version : 1.0
INFO: Stream length : 2797632
INFO: Syncwords : 3238
INFO: Frames : 3238 size 864 bytes
INFO: Frames : 0 size 865 bytes
INFO: Layer : 2
INFO: CRC checksums : no
INFO: Bit rate : 24576 bytes/sec (192 kbit/sec)
INFO: Frequency : 32000 Hz
INFO: Mode : 0 stereo
INFO: Mode extension : 0
INFO: Copyright bit : 0 no copyright
INFO: Original/Copy : 0 copy
INFO: Emphasis : 0 none
INFO: Selecting VCD output profile
INFO: SYSTEMS/PROGRAM stream:
INFO: best-guess multiplexed stream data rate : 1714800
INFO: Setting best-guess data rate.
INFO: New sequence commences...
INFO: sectors muxed: audio=0000000 video=00000000 padding=0000000
**ERROR: Likely buffer under-run at video sector 50
**ERROR: Likely buffer under-run at video sector 50
**ERROR: Likely buffer under-run at video sector 50
**ERROR: Likely buffer under-run at video sector 50
**ERROR: Likely buffer under-run at video sector 50
**ERROR: Likely buffer under-run at video sector 50
**ERROR: Likely buffer under-run at video sector 50
**ERROR: Likely buffer under-run at video sector 50
**ERROR: Likely buffer under-run at video sector 50
**ERROR: Likely buffer under-run at video sector 51
**ERROR: Likely buffer under-run at video sector 51

06-10-2001 22:23:28

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
Looks like you do have a v1.4.1 mplex that I don't!

Could you upgrade your mjpegtools (or, just the mplex from mjpegtools) and try it? I'm usingn the mplex from mjpegtools v1.3b4. Or, is your newer than mine?

06-11-2001 06:28:58

New MessageRE:Problems with ExtractStream. (modified 0 times) nickhull
Profile
It appears that there are other start/restart codes

Try change the following code

if (!PredictSequence ||
TempSequence == 0x205040 ||
TempSequence == 0x205048) // start/restart

to

if (!PredictSequence || (TempSequence & 0xF0F0F0) == 0x205040) // start/restart

It's still a 'guess' but it will work for all the conditions encountered so far.

Nick

06-11-2001 09:01:10

New MessageRE:Problems with ExtractStream. (modified 0 times) nickhull
Profile
Almost forgot.

The 404361 clip sent to me, works fine with the revised code looking for
1) the revised restart code
2) the first video frame not being in the first chunk.

Also, please don't discount the included version of mplex as being 'TiVo' only. The big difference with this version is that it's designed for a single pass through the a/v streams. This makes it named pipes compatible and Linux PC compatible. I've yet to find another mplex that is single pass.

If you try running the included mplex on a PC, rather than TiVo, you get less stuttering on the TiVo and still get good PC playback results.

Try...
ExtractStream piping to netcat
netcat to PC mplex
PC mplex to your viewer of choice.

Nick

06-11-2001 09:10:58

New MessageRE:Problems with ExtractStream. (modified 0 times) Juppers
Profile
I can't seem to get anything I extract and reencode to go past a 10 minute play mark. Here is the output of one such extraction. I have a Sony running 2.0.1-002. All of the recordings I have tried so far are at high quality. I haven't been able to find much discussion other than here about these tools. Any help would be greatly appreciated.

[~]: ExtractStream 569622 569623
Attempting to locate tyStream with fsid 569622...
...tyStream located, sector map follows:
Start Length
0ab08c00 00100000
Block: ab08c00 Count: 1048576 Total blocks 1
/dev/hda10 has 0x00100000 sectors
/dev/hda11 has 0x03606c00 sectors
/dev/hdb2 has 0x00002000 sectors
/dev/hdb3 has 0x098a9900 sectors
Bad chunk, skipping
Sequence number bogus 29cfe0!=21a06c at record 5 of 46
audio failure 0x3b4:0:0xb8 0xb6:0xb1:0x65
Attempting to locate tyStream with fsid 569623...
...tyStream located, sector map follows:
Start Length
0c308c00 00080000
0c388c00 00010000
Block: c308c00 Count: 524288 Total blocks 2
Bad chunk, skippingc00
Block: c388c00 Count: 65536 Total blocks 2
Sequence number bogus 239ac0!=21c12c at record 0 of 45
Sequence number bogus 253c6c!=20e2f8 at record 0 of 31
Sequence number bogus 253c6c!=22d20c at record 0 of 24
Sequence number bogus 253c6c!=24c58c at record 0 of 26
Flushing buffers

06-11-2001 17:12:01

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
Other than the "audio failure", that looks like good output.

Would you say the file size is correct, just your mplexer or video playback application is exiting?

06-11-2001 17:22:22

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
Markb:

>**ERROR: Likely buffer under-run at video sector 50

I upgraded to the latest mjpegtools, and found this error in the source, but I can't repeat it with my a/v files.

Looking at the source, it looks like more of a warning.

How was the resulting video?

06-11-2001 18:06:35

New MessageRE:Problems with ExtractStream. (modified 0 times) Juppers
Profile
The files come out to be about 800 to 900 megs, which would be about right for high quality. The first attempt, the video and audio hung and looped til Media Player crashed. That particular rip seems to have created a shortened file, only about 300 megs. I will try to dump a few more and see if my luck improves. It seems there are only a few I can dump though. Most return pages of the following errors.

Sequence number bogus 236ed8!=2a31bc at record 0 of 45
Sequence number bogus 236ed8!=2c0b54 at record 0 of 43
Sequence number bogus 236ed8!=212654 at record 0 of 53
Sequence number bogus 236ed8!=230230 at record 0 of 43
Sequence number bogus 236ed8!=24e99c at record 0 of 40
Sequence number bogus 236ed8!=26c67c at record 0 of 49
Sequence number bogus 236ed8!=28a320 at record 0 of 40
Sequence number bogus 236ed8!=2a86d8 at record 0 of 50

06-11-2001 18:28:18

New MessageRE:Problems with ExtractStream. (modified 2 times) topset
Profile
>Sequence number bogus 236ed8!=2a31bc at record 0 of 45

The very first of this string of errors is important, as you can see from reading above, we've already got a new sequence restarter: 0x205640... Nick (again, from reading above) thinks the restart indicator should be:

if (!PredictSequence || (TempSequence & 0xF0F0F0) == 0x205040) // start/restart

But I'm not sure we've seen enough cases.

Also, this message is expected a few times... just if you get a continuous string of them, we need to see the first to see a new candidate for sequence restarter.

Are you able to recompile ExtractStream?

------

Media player may not be your best choice for playback. Note the other thread that talks of TMPGEnc.

06-11-2001 19:07:41

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
> I upgraded to the latest mjpegtools, and found this error
> in the source, but I can't repeat it with my a/v files.
>
> Looking at the source, it looks like more of a warning.
>
> How was the resulting video?

Short. It creates an output file that is < 3 megs, while it should be upwards of 24 megs. It seems to play fine, but it's only the first few seconds of the stream.

I found that I don't get these errors when I run mplex with "-m 3" or "-m 4" instead of "-m 2".

-Mark

06-11-2001 19:13:21

New MessageRE:Problems with ExtractStream. (modified 2 times) topset
Profile
>Short. It creates an output file that is < 3 megs,

The "error message" seems to be saying that there's a likelyhood a video or audio packet gap might exist.

Can you try the mplex that was distributed with ExtractStream?

There's a chance that an audio packet that gets cut short might screw it up... if so, I have a fix for that:

--- inptstrm.cpp Sun Jun 10 14:51:17 2001
***************
*** 995,1005 ****

empty_aaunit_struc (&wds->access_unit);

! MACROgetbits(wds->bs, retval, 12);
! if (wds->bs.eobs)
! goto exit3;
!
! if (retval==AUDIO_SYNCWORD)
{
if (!marker_bit(&wds->bs, 1))
goto exit3;
--- 995,1006 ----

empty_aaunit_struc (&wds->access_unit);

! do {
! MACROgetbits(wds->bs, retval, 12);
! if (wds->bs.eobs)
! goto exit3;
! } while
! (retval!=AUDIO_SYNCWORD);
{
if (!marker_bit(&wds->bs, 1))
goto exit3;
***************
*** 1048,1055 ****
audio_info->size_frames[0] = wds->framesize;
audio_info->size_frames = wds->framesize+1;
}
! else
! exit (0);

return 1;
exit3:
--- 1049,1056 ----
audio_info->size_frames[0] = wds->framesize;
audio_info->size_frames = wds->framesize+1;
}
! //else
! // exit (0);

return 1;
exit3:
***************
*** 1123,1140 ****
goto exit3;
}

! wds->offset_bits = bitcount(&wds->bs);
!
! MACROgetbits(wds->bs, retval, 12);
! if (wds->bs.eobs)
! {
! if (!end_bs(&wds->bs))
! goto exit3;
! else
! break;
! }

! if (retval == AUDIO_SYNCWORD)
{
if (!marker_bit(&wds->bs, 1))
goto exit3;
--- 1124,1142 ----
goto exit3;
}

! do {
! wds->offset_bits = bitcount(&wds->bs);

! MACROgetbits(wds->bs, retval, 12);
! if (wds->bs.eobs)
! {
! if (!end_bs(&wds->bs))
! goto exit3;
! else
! break;
! }
! } while
! (retval != AUDIO_SYNCWORD);
{
if (!marker_bit(&wds->bs, 1))
goto exit3;
***************
*** 1171,1178 ****
if (wds->bs.eobs)
goto exit3;
}
! else
! break;
} while (!end_bs(&wds->bs) && !allowRet);
}
/*
--- 1173,1180 ----
if (wds->bs.eobs)
goto exit3;
}
! // else
! // break;
} while (!end_bs(&wds->bs) && !allowRet);
}
/*

(I appologize for this editor: it flattens anything that looks like code)

Otherwise, try the mjpegtools-1.3b4.tar.gz (the last of the series before 1.4).

It seems like they're getting more rigorous concerning their allowed input in 1.4.

>I found that I don't get these errors when I run mplex with "-m 3" or "-m 4" instead of "-m 2".

Do you mean "-f"?

"-m" only takes "1" or "2" as an argument.

06-11-2001 19:55:53

New MessageRE:Problems with ExtractStream. (modified 0 times) markb
Profile | Email
>> -m 3 or -m 4
>Do you mean "-f"? "-m" only takes "1" or "2" as an argument.

Oh, yeah.. that's it. My brain must not be operating at full capicity today! :)

06-11-2001 20:03:45

New MessageRE:Problems with ExtractStream. (modified 0 times) Juppers
Profile
Here is the start of the encoding. It got about 3 pages of this before I stopped it. I currently don't have a tivo compiling environment set up, but was planning on setting one up in a few weeks when I upgrade my linux box to more modern equipment. The old p233 just isn't cutting it for my needs any longer. I do have mjpegtools1.4pre3 installed and an reencoding with that to see if the output lasts more than 10 minutes from the stream I can get to decode.

Attempting to locate tyStream with fsid 422568...
...tyStream located, sector map follows:
Start Length
0b388c00 00080000
00c80000 00080000
Block: b388c00 Count: 524288 Total blocks 2
/dev/hda10 has 0x00100000 sectors
/dev/hda11 has 0x03606c00 sectors
/dev/hdb2 has 0x00002000 sectors
/dev/hdb3 has 0x098a9900 sectors
Bad chunk, skipping
Sequence number bogus 236ed8!=24053c at record 17 of 49
Sequence number bogus 236ed8!=2550b8 at record 5 of 42
audio failure 0x314:0:0x224 0x3a:0x52:0x9b
Sequence number bogus 236ed8!=273304 at record 0 of 44
Sequence number bogus 236ed8!=291674 at record 0 of 52
Sequence number bogus 236ed8!=2ae4cc at record 0 of 41
Sequence number bogus 236ed8!=2cd2bc at record 0 of 34
Sequence number bogus 236ed8!=21f46c at record 7 of 47
audio failure 0x3c4:0:0x244 0xbf:0x4e:0x51
Sequence number bogus 236ed8!=23d258 at record 0 of 60
Sequence number bogus 236ed8!=25a394 at record 0 of 47
Sequence number bogus 236ed8!=277ff4 at record 0 of 33
Sequence number bogus 236ed8!=29724c at record 0 of 50
Sequence number bogus 236ed8!=2b4b0c at record 0 of 62

06-11-2001 20:07:57

New MessageRE:Problems with ExtractStream. (modified 0 times) Juppers
Profile
The seemingly good dump with the audio failure gives this output when reencoding using mplex.

[/tmp/tivo]: mplex -f2 -m 2 -o PROGRAM1.MPEG2 PROGRAM1.M2[AV]
INFO: mplex version 1.4.1 (7.12.2000)
INFO: File PROGRAM1.M2A is a 11172-3 Audio stream.
INFO: File PROGRAM1.M2V is a MPEG-1/2 Video stream.
INFO: Scanning Video stream for access units information.
INFO: VIDEO STREAM:
INFO: nStream length : 787144751
INFO: Sequence start : 3508
INFO: Sequence end : 0
INFO: No. Pictures : 53909
INFO: No. Groups : 3508
INFO: No. I Frames : 3508 avg. size 43699 bytes
INFO: No. P Frames : 14611 avg. size 22024 bytes
INFO: No. B Frames : 35790 avg. size 8718 bytes
INFO: No. D Frames : 0 avg. size 0 bytes
INFO: Horizontal size: 480
INFO: Vertical size : 480
INFO: Aspect ratio : 0.6735
INFO: Picture rate : 29.970 frames/sec
INFO: Bit rate : 3500000 bits/sec
INFO: Computed rate : 3500800 bits/sec
INFO: Peak rate : 5307200 bits/sec
INFO: Vbv buffer size: 229376 bytes
INFO: CSPF : 0
INFO: Scanning Audio stream for access units information.
INFO: Done, stream bit offset 345351168.
INFO: AUDIO STREAM:
INFO: Audio version : 1.0
INFO: Stream length : 43168896
INFO: Syncwords : 49964
INFO: Frames : 49964 size 864 bytes
INFO: Frames : 0 size 865 bytes
INFO: Layer : 2
INFO: CRC checksums : no
INFO: Bit rate : 24576 bytes/sec (192 kbit/sec)
INFO: Frequency : 32000 Hz
INFO: Mode : 0 stereo
INFO: Mode extension : 0
INFO: Copyright bit : 0 no copyright
INFO: Original/Copy : 0 copy
INFO: Emphasis : 0 none
INFO: Selecting VCD output profile
INFO: SYSTEMS/PROGRAM stream:
INFO: best-guess multiplexed stream data rate : 3789600
INFO: Setting best-guess data rate.
INFO: New sequence commences...
INFO: sectors muxed: audio=0000000 video=00000000 padding=0000000
**ERROR: Likely buffer under-run at video sector 14
**ERROR: Likely buffer under-run at video sector 14
**ERROR: Likely buffer under-run at video sector 14
**ERROR: Likely buffer under-run at video sector 14
**ERROR: Likely buffer under-run at video sector 14
**ERROR: Likely buffer under-run at video sector 14
**ERROR: Likely buffer under-run at video sector 14
**ERROR: Likely buffer under-run at video sector 14
**ERROR: Likely buffer under-run at video sector 14
**ERROR: Likely buffer under-run at video sector 15
**ERROR: Likely buffer under-run at video sector 15

[Mon Jun 11 23:09:38 EST]
[/tmp/tivo]: l -l
total 814340
drwxrwxrwx 2 root root 1024 Jun 11 23:09 ./
drwxrwxrwt 13 root root 5120 Jun 11 15:17 ../
-rwxr--r-- 1 nobody nobody 43168896 Jun 12 2001 PROGRAM1.M2A*
-rwxr--r-- 1 nobody nobody 787152916 Jun 12 2001 PROGRAM1.M2V*
-rw-r--r-- 1 root root 292824 Jun 11 23:09 PROGRAM1.MPEG2

06-11-2001 20:16:15

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
>**ERROR: Likely buffer under-run at video sector 14

That's two folks now reporting problems with the 1.4 mplex... but it could easily be caused by ExtractStream:

>Sequence number bogus 236ed8!=24053c at record 17 of 49
>Sequence number bogus 236ed8!=2550b8 at record 5 of 42

This doesn't fit the pattern at all.

Can you post a "ExtractStream -s ... >..." raw dump we can look at? Maybe make a freedrive.com account and post the user/password?

Do this only if this error happens within the first 5MB of raw data.

Otherwise we've got two ExtractStream.c bug fixes since the initial release, and we'll get you a new copy and start from there.

06-11-2001 20:39:29

New MessageRE:Problems with ExtractStream. (modified 0 times) Juppers
Profile
The error happens immediately on decoding for this particular stream. Here is the first 5 megs raw.

http://www.dyslexia.nu/tivo/TEST.RAW

Currently I am trying to get the toolchain working but it is giving me some troubles. I will let you know when I can compile locally.

06-11-2001 20:54:14

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
Got it. Nick works nights, he's got the tools built to analyze it. He'll probably have a response by morning.
06-11-2001 21:18:54

New MessageRE:Problems with ExtractStream. (modified 0 times) nickhull
Profile
Juppers. Thanks for the raw dump.

In ExtractStream.cpp you can try placing a

if (!foundfirstframe) return;

after the end of the first 'for loop' in parse_chunk.
It should solve this, and some other problems - what was the clip anyway? SNL or something?

06-12-2001 07:53:39

New MessageRE:Problems with ExtractStream. (modified 0 times) nickhull
Profile
JUPPERS. Please send email to nickhull@yahoo.com
06-12-2001 08:29:29

New MessageRE:Problems with ExtractStream. (modified 1 times) Juppers
Profile
Ignore this. I had my mount permissions wrong and I can't find a way to delete this one. :)

I have been testing this updated version, and I keep getting an error I was getting occasionally with the previous version. I haven't figured out what causes this yet, as it happens randomly. Any clues would be helpful.

Sometimes it works, other times it gives this.

[/var/hack]: ExtractStream 422568 422569
Attempting to locate tyStream with fsid 422568...
...tyStream located, sector map follows:
Start Length
0b388c00 00080000
00c80000 00080000
Failed to connect to Video codec.

06-12-2001 14:50:36

New MessageRE:Problems with ExtractStream. (modified 0 times) Juppers
Profile
New vs Old. Still multiplex the old stream back together and about to watch the new one and see how it turned out.

[Tue Jun 12 22:38:23 EST]
[~]: NewExtractStream 365306 365307
Attempting to locate tyStream with fsid 365306...
...tyStream located, sector map follows:
Start Length
06308c00 00100000
Block: 6308c00 Count: 1048576 Total blocks 1
Playing block 06308c00/dev/hda10 has 0x00100000 sectors
/dev/hda11 has 0x03606c00 sectors
/dev/hdb2 has 0x00002000 sectors
/dev/hdb3 has 0x098a9900 sectors
Bad chunk, skipping
Playing block 06408c00Bad chunk, skipping
Attempting to locate tyStream with fsid 365307...
...tyStream located, sector map follows:
Start Length
06508c00 00080000
06588c00 00010000
06598c00 00004000
Block: 6508c00 Count: 524288 Total blocks 3
Playing block 06508c00Bad chunk, skipping
Block: 6588c00 Count: 65536 Total blocks 3
Playing block 06588c00Sequence number bogus 23a338!=21bfb8 at record 0 of 44
Block: 6598c00 Count: 16384 Total blocks 3
Playing block 06598c00Sequence number bogus 2ae748!=28ffa8 at record 0 of 40
Playing block 06599400Sequence number bogus 2b391c!=2ac864 at record 0 of 23
Playing block 06599500Sequence number bogus 2b391c!=2cbbe4 at record 0 of 27
Playing block 06599600Sequence number bogus 2b391c!=21e2bc at record 0 of 26
Playing block 06599700Sequence number bogus 2b391c!=23d608 at record 0 of 33
Playing block 06599800Sequence number bogus 2b391c!=25bad0 at record 0 of 39
Playing block 06599900Sequence number bogus 2b391c!=279848 at record 0 of 27
Playing block 06599a00Sequence number bogus 2b391c!=298b80 at record 0 of 19
Playing block 06599b00Sequence number bogus 2b391c!=2b7f54 at record 0 of 36
Playing block 06599c00Sequence number bogus 2b391c!=2097dc at record 0 of 15
Playing block 06599d00Sequence number bogus 2b391c!=22963c at record 0 of 25
Playing block 06599e00Sequence number bogus 2b391c!=2482dc at record 0 of 35
Playing block 06599f00Sequence number bogus 2b391c!=2666e8 at record 0 of 28
Playing block 0659a000Sequence number bogus 2b391c!=285a20 at record 0 of 32
Playing block 0659a100Sequence number bogus 2b391c!=2a424c at record 0 of 27
Playing block 0659a200Sequence number bogus 2b391c!=2c3524 at record 0 of 30
Playing block 0659a300Sequence number bogus 2b391c!=215200 at record 0 of 31
Playing block 0659a400Sequence number bogus 2b391c!=233aac at record 0 of 33
Playing block 0659a500Sequence number bogus 2b391c!=2522d4 at record 0 of 24
Playing block 0659a600Sequence number bogus 2b391c!=2715d0 at record 0 of 23
Playing block 0659a700Sequence number bogus 2b391c!=29095c at record 0 of 35
Playing block 0659a800Sequence number bogus 2b391c!=2ae728 at record 0 of 23
Playing block 0659a900Sequence number bogus 2b391c!=2cdaa8 at record 0 of 24
Playing block 0659aa00Sequence number bogus 2b391c!=21fe58 at record 0 of 35
Playing block 0659ab00Sequence number bogus 2b391c!=23e9bc at record 0 of 24
Playing block 0659ac00Sequence number bogus 2b391c!=25d9cc at record 0 of 37
Playing block 0659ad00Sequence number bogus 2b391c!=27b758 at record 0 of 23
Playing block 0659ae00Sequence number bogus 2b391c!=29aae8 at record 0 of 33
Playing block 0659af00Sequence number bogus 2b391c!=2b9310 at record 0 of 20
Playing block 0659b000Sequence number bogus 2b391c!=20c1a0 at record 0 of 25
Playing block 0659b100Sequence number bogus 2b391c!=22add0 at record 0 of 27
Playing block 0659b200Sequence number bogus 2b391c!=249d90 at record 0 of 35
Playing block 0659b300Sequence number bogus 2b391c!=268594 at record 0 of 33
Playing block 0659b400Sequence number bogus 2b391c!=286384 at record 0 of 24
Playing block 0659b500Sequence number bogus 2b391c!=2a56f4 at record 0 of 20
Playing block 0659b600Sequence number bogus 2b391c!=2c4a48 at record 0 of 38
Playing block 0659b700Sequence number bogus 2b391c!=21629c at record 0 of 22
Flushing buffers59cc00

[Tue Jun 12 23:25:41 EST]
[~]: ExtractStream 365306 365307
Attempting to locate tyStream with fsid 365306...
...tyStream located, sector map follows:
Start Length
06308c00 00100000
Block: 6308c00 Count: 1048576 Total blocks 1
/dev/hda10 has 0x00100000 sectors
/dev/hda11 has 0x03606c00 sectors
/dev/hdb2 has 0x00002000 sectors
/dev/hdb3 has 0x098a9900 sectors
Bad chunk, skipping
Bad chunk, skippingc00
Attempting to locate tyStream with fsid 365307...
...tyStream located, sector map follows:
Start Length
06508c00 00080000
06588c00 00010000
06598c00 00004000
Block: 6508c00 Count: 524288 Total blocks 3
Bad chunk, skippingc00
Block: 6588c00 Count: 65536 Total blocks 3
Sequence number bogus 23a338!=21bfb8 at record 0 of 44
Block: 6598c00 Count: 16384 Total blocks 3
Sequence number bogus 2ae748!=28ffa8 at record 0 of 40
Sequence number bogus 2b391c!=2ac864 at record 0 of 23
Sequence number bogus 2b391c!=2cbbe4 at record 0 of 27
Sequence number bogus 2b391c!=21e2bc at record 0 of 26
Sequence number bogus 2b391c!=23d608 at record 0 of 33
Sequence number bogus 2b391c!=25bad0 at record 0 of 39
Sequence number bogus 2b391c!=279848 at record 0 of 27
Sequence number bogus 2b391c!=298b80 at record 0 of 19
Sequence number bogus 2b391c!=2b7f54 at record 0 of 36
Sequence number bogus 2b391c!=2097dc at record 0 of 15
Sequence number bogus 2b391c!=22963c at record 0 of 25
Sequence number bogus 2b391c!=2482dc at record 0 of 35
Sequence number bogus 2b391c!=2666e8 at record 0 of 28
Sequence number bogus 2b391c!=285a20 at record 0 of 32
Sequence number bogus 2b391c!=2a424c at record 0 of 27
Sequence number bogus 2b391c!=2c3524 at record 0 of 30
Sequence number bogus 2b391c!=215200 at record 0 of 31
Sequence number bogus 2b391c!=233aac at record 0 of 33
Sequence number bogus 2b391c!=2522d4 at record 0 of 24
Sequence number bogus 2b391c!=2715d0 at record 0 of 23
Sequence number bogus 2b391c!=29095c at record 0 of 35
Sequence number bogus 2b391c!=2ae728 at record 0 of 23
Sequence number bogus 2b391c!=2cdaa8 at record 0 of 24
Sequence number bogus 2b391c!=21fe58 at record 0 of 35
Sequence number bogus 2b391c!=23e9bc at record 0 of 24
Sequence number bogus 2b391c!=25d9cc at record 0 of 37
Sequence number bogus 2b391c!=27b758 at record 0 of 23
Sequence number bogus 2b391c!=29aae8 at record 0 of 33
Sequence number bogus 2b391c!=2b9310 at record 0 of 20
Sequence number bogus 2b391c!=20c1a0 at record 0 of 25
Sequence number bogus 2b391c!=22add0 at record 0 of 27
Sequence number bogus 2b391c!=249d90 at record 0 of 35
Sequence number bogus 2b391c!=268594 at record 0 of 33
Sequence number bogus 2b391c!=286384 at record 0 of 24
Sequence number bogus 2b391c!=2a56f4 at record 0 of 20
Sequence number bogus 2b391c!=2c4a48 at record 0 of 38
Sequence number bogus 2b391c!=21629c at record 0 of 22
Sequence number bogus 2b391c!=23564c at record 0 of 31
Sequence number bogus 2b391c!=254204 at record 0 of 38
Sequence number bogus 2b391c!=2726bc at record 0 of 14
Sequence number bogus 2b391c!=291e60 at record 0 of 27
Sequence number bogus 2b391c!=2b0dcc at record 0 of 38
Sequence number bogus 2b391c!=2cf590 at record 0 of 23
Sequence number bogus 2b391c!=221d10 at record 0 of 27
Sequence number bogus 2b391c!=2405b0 at record 0 of 35
Sequence number bogus 2b391c!=25ee18 at record 0 of 30
Sequence number bogus 2b391c!=27d674 at record 0 of 29
Sequence number bogus 2b391c!=29c5bc at record 0 of 22
Sequence number bogus 2b391c!=2bbcc8 at record 0 of 34
Sequence number bogus 2b391c!=20d574 at record 0 of 41
Sequence number bogus 2b391c!=22b2c8 at record 0 of 21
Sequence number bogus 2b391c!=24a678 at record 0 of 21
Sequence number bogus 2b391c!=269a1c at record 0 of 21
Sequence number bogus 2b391c!=289498 at record 0 of 27
Sequence number bogus 2b391c!=2a80a4 at record 0 of 35
Sequence number bogus 2b391c!=2c653c at record 0 of 40
Sequence number bogus 2b391c!=2176a4 at record 0 of 32
Sequence number bogus 2b391c!=236978 at record 0 of 26
Flushing buffers

06-12-2001 17:07:51

New MessageRE:Problems with ExtractStream. (modified 0 times) Juppers
Profile
Video hung in exact spot as before with the new version. About 15 minutes into the program. Annoyingly enough on a Tom Green scream. This was using tmpg as the encoder playing the resultant file through widows media player.
06-12-2001 17:18:48

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
Sorry, I forgot to tell you that the hang at the end was a known bug when I gave it to you... I was just trying to make sure all the other bugs were fixed. If you write to A/V files rather than pipe to mplex, then process the files w/ mplex, the hang shouldn't be there.
06-12-2001 20:28:59

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
>Video hung in exact spot as before with the new version.

Was ExtractStream hanging with the old version?

>About 15 minutes into the program.

The "flushing buffers" message says it's completed all the tyStreams (except for the last 2MB). Should there have been lots more video? If so, use the httpd.tcl, select the "mfs" option, traverse to this show, and look at the "parts" -- does it show the same parts as the "NowShowing" option?

>This was using tmpg as the encoder playing the resultant file through widows media player.

So you were piping directly from mplex into tmpg and Windows Media Player (no intermediate file)? If so, how was the performance? What was the quality setting? Can you provide instructions?

06-12-2001 20:42:24

New MessageRE:Problems with ExtractStream. (modified 0 times) Juppers
Profile
I extracted the show into program1.m2v and m2a, I then used tmpg to multiplex the pieces back together into program1.mpg. When playing program1.mpg, it hung at the same spot as it did when I extracted the program with the first release of ExtractStream, the only difference is this time around I could use the scroll bar to advance the video. I am going to multiplex them again using the mplex I was given today and the one I compiled with the mjpegtools. I will report back if there are any differences.
06-12-2001 20:49:03

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
>When playing program1.mpg, it hung at the same spot as it did when I extracted the program with the first release of ExtractStream, the only difference is this time around I could use the scroll bar to advance the video.

Windows media player found a bad spot it couln't get around with the first ExtractStream, but could get around with the new one (even though it still stopped there)?

This might be the partial audio packet bug... Try one of the mplex fixes for mjpegtools in ExtractStream.html, or for mplex as listed somewhere in this thread.

06-12-2001 21:48:55

New MessageRE:Problems with ExtractStream. (modified 0 times) zobetron
Profile
What is the most recent version of ExtractStream? Is it still ExtractStream-0.1.tgz? If not where can I get the most recent?

TIA.

06-12-2001 22:29:35

New MessageRE:Problems with ExtractStream. (modified 0 times) topset
Profile
>What is the most recent version of ExtractStream? Is it still ExtractStream-0.1.tgz? If not where can I get the most recent?

A new one should be out soon, but this is current.

06-13-2001 07:14:55

New MessageRE:Problems with ExtractStream. (modified 1 times) foundation
Profile
Is anybody still reading this forum? What this thread needs is one more response so here goes...

I've got extractstream working and it's great. My Tivo does a much higher quality compression per bitrate than my All in Wonder Radeon (using the gomotion sdk from ligos for real time mpeg2)

So I decided to grab all my saved recordings off my tivo. When I went to grab a recording from a long time ago (1/31 -- pre 2.0.1) It gave me a ton of the Sequence number bogus errors.

When I look at the finished file it is 1053KB for the video and 75029KB for the audio. The audio is fine. The video is not recognized by media player or tmpgenc. I'm running 2.01 now, am I having problems because it was recorded under the old kernel or has it gotten corrupted? The video and audio play fine on the tivo (it was on the high quality setting).

Any suggestions?

Thanks,
Foundation

Here's the record from MFS I'm trying to extract.

Recording 932054/10 {
Version = 74
Showing = 932054/11
RecordQuality = 75
Score = 0
BitRate = 0
Part = 932054/16
State = 4
StartDate = 11354
StartTime = 14398
StopDate = 11354
StopTime = 17999
ExpirationDate = 24855
ExpirationTime = 0
SelectionType = 3
DeletionTime = 0
DeletionDate = 0
IndexUsed = 932054/11
IndexPath = /Recording/Active/1:11354:14398:932054 /Recording/NowShowing/A:88645:85601:932054 /Recording/NoReRecord/3196828:NN:99999:00932054
}

RecordingPart 932054/16 {
File = 957301
Begin = -2000
End = 3599000
}
Showing 932054/11 {
Program = 929784/-1
Station = 2303/-1
Date = 11354
Time = 14400
Duration = 3600
TvRating = 5
Bits = 786435
DontIndex = 1
Reason = 3
Genre = 35 1005
SeriesId = 11485
IndexUsedBy = 932054/10
}


and a log of the first few errors. The complete log is available at http://65.0.171.59.:8080/extractstream.log for a while.

/var/hack/ExtractStream 957301
Attempting to locate tyStream with fsid 957301...
...tyStream located, sector map follows:
Start Length
00d00000 00200000
00f00000 00100000
00140000 00020000
Block: d00000 Count: 2097152 Total blocks 3
Playing block 00d00000/dev/hda10 has 0x00100000 sectors
/dev/hda11 has 0x017a1400 sectors
Bad chunk, skipping
Playing block 00d00100No first frame, skipping block
Playing block 00d00200No first frame, skipping block
Playing block 00d00300No first frame, skipping block
Playing block 00d00400No first frame, skipping block
Playing block 00d00500No first frame, skipping block
Playing block 00d00600No first frame, skipping block
Playing block 00d00700No first frame, skipping block
Playing block 00d00800Sequence number bogus fabd73!=ff731f at record 16 of 36
Playing block 00d00900Sequence number bogus fabd73!=d3bd1f at record 10 of 35
Playing block 00d00a00Sequence number bogus fabd73!=ed7d1f at record 9 of 33
Playing block 00d00b00Sequence number bogus fabd73!=c9fd1f at record 8 of 38
Playing block 00d00c00Sequence number bogus fabd73!=8dbd1f at record 9 of 25
Playing block 00d00d00Sequence number bogus fabd73!=ffb41f at record 8 of 34
Playing block 00d00e00Sequence number bogus fabd73!=92fd1f at record 9 of 28
Playing block 00d00f00Sequence number bogus fabd73!=c6fd1f at record 10 of 37
Playing block 00d01000Sequence number bogus fabd73!=af3d1f at record 9 of 36
Playing block 00d01100Sequence number bogus fabd73!=693d1f at record 8 of 30
Playing block 00d01200Sequence number bogus fabd73!=a8bd1f at record 8 of 34
Playing block 00d01300Sequence number bogus fabd73!=aa7d1f at record 8 of 36
Playing block 00d01400Sequence number bogus fabd73!=9e3d1f at record 10 of 29
Playing block 00d01500Sequence number bogus fabd73!=ffbd1f at record 11 of 41
Playing block 00d01600Sequence number bogus fabd73!=237d1f at record 4 of 17
Playing block 00d01700Sequence number bogus fabd73!=a1bd1f at record 9 of 38
Playing block 00d01800Sequence number bogus fabd73!=d93d1f at record 10 of 38
Playing block 00d01900Sequence number bogus fabd73!=ffb41f at record 12 of 51
Playing block 00d01a00Sequence number bogus fabd73!=93bd1f at record 8 of 29
Playing block 00d01b00Sequence number bogus fabd73!=dfb51f at record 8 of 30
Playing block 00d01c00Sequence number bogus facc57!=ffbd1f at record 12 of 34
Playing block 00d01d00Sequence number bogus facc57!=ffbd1f at record 7 of 23
Playing block 00d01e00Sequence number bogus facc57!=5ffd1f at record 12 of 40
Playing block 00d01f00Sequence number bogus facc57!=8cbd1f at record 6 of 26
Playing block 00d02000Sequence number bogus facc57!=cafd1f at record 10 of 33
Playing block 00d02100Sequence number bogus facc57!=bd3d1f at record 9 of 35
Playing block 00d02200Sequence number bogus facc57!=ffbd1f at record 8 of 32
Playing block 00d02300Sequence number bogus facc57!=de3d1f at record 11 of 34
Playing block 00d02400Sequence number bogus facc57!=cb3d1f at record 10 of 45
Playing block 00d02500Sequence number bogus facc57!=323d1f at record 6 of 21
Playing block 00d02600Sequence number bogus facc57!=73fd1f at record 12 of 40
Playing block 00d02700Sequence number bogus facc57!=ffbd1f at record 8 of 26
Playing block 00d02800Sequence number bogus facc57!=867d1f at record 10 of 34
Playing block 00d02900Sequence number bogus facc57!=ffbd1f at record 11 of 33
Playing block 00d02a00Sequence number bogus facc57!=ebfd1f at record 8 of 39
Playing block 00d02b00Sequence number bogus facc57!=667d1f at record 10 of 34
Playing block 00d02c00Sequence number bogus facc57!=e03d1f at record 7 of 28

Log continues for 992KB

07-13-2001 17:18:48

Reply to Thread | Printer |
All times are PSTPowered by UltraBoard v1.62



Copyright © 2000, Netmake Inc. All Rights Reserved.
See Terms and Conditions for more information.




i-opener opener laptop notebook computer help drivers dll free windows dos repair fix linux mac macintosh 2000 95 98 nt pc configure hardware software sound video netscape explorer network networking lan wan software cmos fat bios printer card mouse modem ide scsi cd rom controllers scanner tape hard drive cgi scripts source code mp3