Digiguide grabber detail in 3.10.00W

Discussion of Version 3 and developing lua grabbers
alanbirtles
Site Admin
Posts: 495
Joined: Mon Sep 06, 2004 4:44 pm

Re: Digiguide grabber detail in 3.10.00W

Post by alanbirtles » Tue Jun 03, 2008 6:10 pm

try just using localhost as the ip address

bedfordcf
Posts: 3
Joined: Wed Jun 04, 2008 9:26 am

Re: Digiguide grabber detail in 3.10.00W

Post by bedfordcf » Wed Jun 04, 2008 9:33 am

I thought I'd set this up correctly but when I run it, it seems to download 15Mb of data, then fails with a http protocol error. I can use the url to see the data and running DG2XML returns data ok. Any ideas ?

thanks

alanbirtles
Site Admin
Posts: 495
Joined: Mon Sep 06, 2004 4:44 pm

Re: Digiguide grabber detail in 3.10.00W

Post by alanbirtles » Wed Jun 04, 2008 10:03 am

any more details of the error? a log would help

bedfordcf
Posts: 3
Joined: Wed Jun 04, 2008 9:26 am

Re: Digiguide grabber detail in 3.10.00W

Post by bedfordcf » Wed Jun 04, 2008 10:26 am

sorry, log attached
Attachments
log.zip
(447 Bytes) Downloaded 2017 times

alanbirtles
Site Admin
Posts: 495
Joined: Mon Sep 06, 2004 4:44 pm

Re: Digiguide grabber detail in 3.10.00W

Post by alanbirtles » Thu Jun 05, 2008 7:33 pm

all the fields should now be included. the http error messages should also be more detailed

b00sfuk
Posts: 31
Joined: Wed Dec 14, 2005 11:18 am

Re: Digiguide grabber detail in 3.10.00W

Post by b00sfuk » Thu Jun 05, 2008 7:56 pm

Alan, great thanks. I'll install it later and feedback.
Thanks again for your time on this

b00sfuk
Posts: 31
Joined: Wed Dec 14, 2005 11:18 am

Re: Digiguide grabber detail in 3.10.00W

Post by b00sfuk » Thu Jun 05, 2008 9:27 pm

Alan, gettting a problem with the grabber. I get no data apart from the header in the xml file but the log file contains all the XML data format with this error below (which does not seem to informative). Any ideas please? Is it me? It worked OK for me with the previous version when I tested it.

thanks again
Simon

Code: Select all

21:05:49 - Starting pass 1 of grabber UK_Digi
21:05:49 - Logging In
21:05:49 - Getting start date
21:05:49 - Downloading http://localhost:2402/skin-DG2XML/sid-53F6AABC-DDED-4949-A06D-FED9A63ED636/viewmultichannelgrid?htd=3000&dt=1212091549&chi=0&noc=3000
21:06:46 - HTTP Protocol error(-1) downloading "http://localhost:2402/skin-DG2XML/sid-53F6AABC-DDED-4949-A06D-FED9A63ED636/viewmultichannelgrid?htd=3000&dt=1212091549&chi=0&noc=3000" : <?xml version="1.0" encoding="ISO-8859-1"?>
<tv generator-info-name="digiguide">
<channel id="101"><display-name><![CDATA[BBC 1 Midlands]]></display-name></channel><programme channel="101" start="05/29/2008 21:00" stop="05/29/2008 22:00">
	<title><![CDATA[The Invisibles]]></title>
	<desc><![CDATA[Comedy drama series about two robbers drawn out of retirement. Young Nick's dad is in trouble and the boys promise to help. But, when Hedley discovers he's going to be a father, he starts to question his role in the gang - with devastating consequences.]]></desc>
	<category><![CDATA[Drama]]></category>
	<length>1 hour</length>
	<series-number><![CDATA[]]></series-number>
	<episode-number><![CDATA[]]></episode-number>
	<episode-name><![CDATA[]]></episode-name>
	<miscellaneous><![CDATA[Stereo, Widescreen, Subtitles]]></miscellaneous>
	<credits>
		
		<credit type="Starring:"><![CDATA[Anthony Head]]><![CDATA[Paul Barber]]><![CDATA[Warren Clarke]]><![CDATA[Dean Lennox Kelly]]><![CDATA[Jenny Agutter]]><![CDATA[Mina Anwar]]></credit>
		
		
		
		
	</credits>

.............repetitive entries for all programmes removed

21:08:23 - Error downloading listings (-1)
21:08:23 - Grab failed during start function of pass 1
21:08:23 -    0 programs found
21:08:23 - Combining Grabber Programs
21:08:23 -    Finished Combining Grabber Programs
21:08:23 - Writing XML
21:08:23 -    XML Written

alanbirtles
Site Admin
Posts: 495
Joined: Mon Sep 06, 2004 4:44 pm

Re: Digiguide grabber detail in 3.10.00W

Post by alanbirtles » Thu Jun 05, 2008 9:33 pm

the only thing i have changed is the number of channels, i increased it from 1000 to 3000, try changing the last number in this line in the grabber script:

Code: Select all

URL = URL.."skin-DG2XML/"..sessionID.."/viewmultichannelgrid?htd=3000&dt="..startDate.."&chi=0&noc=3000"

b00sfuk
Posts: 31
Joined: Wed Dec 14, 2005 11:18 am

Re: Digiguide grabber detail in 3.10.00W

Post by b00sfuk » Thu Jun 05, 2008 9:35 pm

Scrub the above. I tried reinstalling XMLTV (I had previously installed over top of RT install) and with this fresh install it worked OK. Weird.
Now the test it in GB-PVR
thanks for quick reply but I did not need it (yet).

alanbirtles
Site Admin
Posts: 495
Joined: Mon Sep 06, 2004 4:44 pm

Re: Digiguide grabber detail in 3.10.00W

Post by alanbirtles » Thu Jun 05, 2008 9:47 pm

i think digiguide may be running out of memory. i could try downloading the data in small chunks instead of all at once

b00sfuk
Posts: 31
Joined: Wed Dec 14, 2005 11:18 am

Re: Digiguide grabber detail in 3.10.00W

Post by b00sfuk » Thu Jun 05, 2008 10:05 pm

I think I've narrowed it down to the timeouts so memory might not be an issue. I had put in longer timeouts (10000) and I got the error, if I left it at default it worked fine (which is why my reinstall fixed it). I now get 3 read timeouts but it completes OK (I can try small increases if necessary I guess).

On another matter why is it that the channel numbers have a prefix in the XML file. E.g. rather than the preferred "101" I get "UK_Digi_101". This messes up the import into GB-PVR and I cannot seem to get the altchids.txt file to overide it.

alanbirtles
Site Admin
Posts: 495
Joined: Mon Sep 06, 2004 4:44 pm

Re: Digiguide grabber detail in 3.10.00W

Post by alanbirtles » Thu Jun 05, 2008 10:09 pm

the latest svn of the grabber now downloads 5 channels at a time and may be more reliable.

alanbirtles
Site Admin
Posts: 495
Joined: Mon Sep 06, 2004 4:44 pm

Re: Digiguide grabber detail in 3.10.00W

Post by alanbirtles » Thu Jun 05, 2008 10:21 pm

thats the way that all grabbers work by default. the grabber's id is appended to all of the channel ids to ensure that the ids are all unique. a simple post processor will fix it:

Code: Select all

require("XGUI_Lib")

local function Pass1Func(grabber,channel)
	XGUI.SetStatus(2,"Processing "..XGUI_Lib.XS.Val(channel.Name))
	channel.XML_ID=channel.ID
	return channel
	end

PostProcs.UK_Digi_ALT_Digi={
	Name = "UK - Digiguide Standard Channel IDS",
	Version = "1.00.00",
	Author = "Alan Birtles",
	InfoURL = "http://www.birtles.org.uk/xmltv/wiki/index.php?title=UK_Digi_ALT_Digi",
	Grabber = "UK_Digi",
	Passes={
		Pass1={
			Type=XGUI.PassTypes.Channel,
			Func=Pass1Func,
			},
		},
	}

b00sfuk
Posts: 31
Joined: Wed Dec 14, 2005 11:18 am

Re: Digiguide grabber detail in 3.10.00W

Post by b00sfuk » Thu Jun 05, 2008 10:32 pm

Alan, brilliant. That works a treat for the channel numbers. I'm not having any reliability problems now but will try to test the svn build tomorrow when I get time.
thanks again for the support.

bedfordcf
Posts: 3
Joined: Wed Jun 04, 2008 9:26 am

Re: Digiguide grabber detail in 3.10.00W

Post by bedfordcf » Thu Jun 05, 2008 11:17 pm

I'm still getting the protocol error as described by b00sfuk at 9.27. If I lower the number of channels to 20 in the grabber script it works, any higher I get the error. Will await the next release which grabs 5 channels at a time.

thanks.

Post Reply