I don't know what the deal is, but I keep getting the following error doing 'drush -u admin -d rets-import'

Undefined index:  MLnumber drealty.daemon.php:540 [5.42 sec, 54.57 MB]                                                                                           [notice]
Undefined index:  Matrix_Unique_ID drealty.daemon.php:540 [5.42 sec, 54.57 MB]                                                                                   [notice]
Undefined index:  Status drealty.daemon.php:540 [5.42 sec, 54.57 MB] 

This scrolls on forever...

Notes:

  • I was able to get version 7.x-2.0-beta18 to import data.
  • with version 3.0, I tried switching out the MLnumber and Matrix_Unique_ID in 'Id Field' and 'key field'; i.e. doing all possible combinations and each time the same error message.

Comments

duntuk’s picture

here's a full metadata listing:

note: 'MLS_ID' isn't the actual MLS # it's just the 'type' of MLS which has predefined values. I mistaken that initially. 'MLnumber' is the actual MLS #. 'Matrix_Unique_ID' is defined as a 'Key Field' .



RETS Server:	MRMLS RETS System
RETS System ID:	MRMLS


Resource:	Property
Class:	listing_mrmls_resi
# of Fields:	165
Key Field:	Matrix_Unique_ID 
Object Types:	View Object Types
	Export Field Data
Download the metadata in CSV format

SystemName	StandardName	Description	Type	Length	Lookup
220vLocation 		220 Voltage Location	Character	122	Values
Acres 		Acres	Decimal	9	
AdNumber 		AdNumber	Character	15	
APN 		Assessor's Parcel Number	Character	19	
Appliances 		Kitchen Appliances	Character	150	Values
Area 		Area	Character	60	Values
Assessments 		Assessments	Character	68	Values
Association 		Association	Character	173	Values
AssociationDues1 		Association Dues 1	Int	0	
AssociationDues1Frequency 	AssociationDuesFrequency	AssociationDues1Frequency	Character	12	Values
AssociationDues2 		Association Dues 2	Int	0	
AssociationDues2Frequency 	AssociationDuesFrequency	AssociationDues2Frequency	Character	12	Values
AssociationYN 	AssociationYN	AssociationYN	Boolean	1	
AttachedStructure 		Attached/Detached	Character	60	Values
BasementSqft 		BasementSqft	Int	10	
BathsFull 	BATHFULL	BathFull	Int	25	
BathsHalf 	BATHHALF	BathHalf	Int	25	
BathsOqtr 	BATHOQTR	BathOqtr	Int	25	
BathsTotal 	BATHS	Number of Bathrooms	Decimal	5	
BathsTqtr 	BATHTQTR	BathTqtr	Int	25	
Bedrooms 	BEDS	Number of Bedrooms	Int	25	
BlockNumber 		Block Number	Character	4	
BuildersTractCode 	BuildersTractCode	BuildersTractCode	Character	50	
BuildersTractName 	BuildersTractName	BuildersTractName	Character	50	
CDOM 		CDOM	Int	0	
City 	CITY	City	Character	60	Values
CoLA_Board 		cola_primaryboard	Character	1	Values
CoLA_DreLicenseNumber 	Co LA Dre License Number	Co LA Dre License Number	Character	20	
CoLA_FirstName 		cola_fname	Character	20	
CoLA_LastName 		cola_lname	Character	20	
CoLA_PublicID 		cola_publicid	Character	30	
CoLo_Code 		colo_code	Character	30	
CoLo_Name 		colo_name	Character	30	
Cookery 		Kitchen Range/Oven	Character	60	Values
Cooling 		Air Conditioning Description	Character	75	Values
CoSA_Board 		CoSA_Board	Character	1	Values
CoSA_DreLicenseNumber 	Co SA Dre License Number	Co SA Dre License Number	Character	20	
CoSA_FirstName 		CoSA_FirstName	Character	20	
CoSA_LastName 		CoSA_LastName	Character	20	
CoSO_Code 		CoSO_Code	Character	30	
CoSO_DreLicenseNumber 	CoSO_DreLicenseNumber	CoSO_DreLicenseNumber	Character	20	
CoSO_Name 		CoSO_Name	Character	30	
Country 		Country	Character	60	Values
County 		County	Character	60	Values
CrossStreets 		Cross Streets	Character	30	
DateCanceled 		DateCanceled	DateTime	23	
DateClosedSale 		DateClosedSale	DateTime	23	
DateEnding 		DateEnding	DateTime	23	
DateHoldActivation 		DateHoldActivation	DateTime	23	
DateListing 	LISTDATE	Listing Date	DateTime	23	
DateModified 	MODIFICATIONTIMESTAMP	Transaction Date	DateTime	23	
DatePending 		Pending Date	DateTime	23	
DatePhotoModified 		photomodifieddate	DateTime	23	
DateSold 		Close of Escrow/End Date	DateTime	23	
DocumentNumber 		DocumentNumber	Character	50	
DOM 	DAYSONMARKET	Active Days on the Market	Int	25	
DrivingDirections 		DrivingDirections	Character	250	
EatingArea 		Eating Area	Character	250	Values
Elevation 		Elevation Above Sea Level	Character	9	
EntryLocation 	EntryLocation	EntryLocation	Character	400	Values
Exterior 		Exterior	Character	108	Values
Fence 		Fence	Character	236	Values
Fireplace 		Fireplace	Character	88	Values
Floor 		Floor	Character	274	Values
GarageAttached 		Garage Description	Character	60	Values
GreenBuildingCertification 	GreenBuildingCertification	GreenBuildingCertification	Character	60	Values
GreenCertificationRating 	GreenCertificationRating	GreenCertificationRating	Decimal	6	
GreenCertifyingBody 	GreenCertifyingBody	GreenCertifyingBody	Character	50	
GreenEnergyEfficient 	GreenEnergyEfficient	GreenEnergyEfficient	Character	400	Values
GreenEnergyGeneration 	GreenEnergyGeneration	GreenEnergyGeneration	Character	400	Values
GreenIndoorAirQuality 	GreenIndoorAirQuality	GreenIndoorAirQuality	Character	400	Values
GreenLocation 	GreenLocation	GreenLocation	Character	400	Values
GreenSustainability 	GreenSustainability	GreenSustainability	Character	400	Values
GreenWalkScore 	GreenWalkScore	GreenWalkScore	Int	10	
GreenWaterConservation 	GreenWaterConservation	GreenWaterConservation	Character	400	Values
GreenYearCertified 	GreenYearCertified	GreenYearCertified	Int	10	
Heating 	HEATING	Heat	Character	274	Values
Interior 		Interior	Character	442	Values
InternetSendAddressYN 		Send Address To Internet	Boolean	5	
InternetSendListingYN 		Free Internet Advertisement	Boolean	5	
LA_Board 		laprimaryboard	Character	1	Values
LA_DreLicenseNumber 	LA Dre License Number	LA Dre License Number	Character	20	
LA_FirstName 		lafname	Character	20	
LA_LastName 		lalname	Character	20	
LA_PublicID 		Listing Agent Public ID	Character	30	
LandFeeLease 		Land Fee/Lease	Character	60	Values
Laundry 		Laundry	Character	110	Values
LeaseConsideredYN 		Offered For Lease	Boolean	5	
ListPrice 	LISTPRICE	List Price	Int	0	
ListPriceExcludes 		List Price Excludes	Character	250	
ListPriceIncludes 	ListPriceIncludes	ListPriceIncludes	Character	250	
LO_Code 	LISTOFFICEBROKERID	Listing Office Code	Character	30	
LO_Name 	LISTOFFICENAME	Listing Office Name	Character	30	
LotCommunity 		LotCommunity	Character	656	Values
LotDimensions 		Lot Dimensions	Character	40	
LotNumber 		Lot Number	Character	4	
LotSquareFootage 		Lot Square Footage	Int	0	
Make 		Mobile Home Make	Character	40	
Matrix_Unique_ID  		Matrix_Unique_ID	Long	12	
MLnumber 	LISTINGID	Multiple Listing Number	Character	12	
MLS_ID 		MLS ID	Character	60	Values
ModelCode 		Model Code	Character	1	
ModelName 		Model Name	Character	14	
NumberCarportSpaces 		Number of Carport Spaces	Int	25	
NumberGarageSpaces 		Number of Garage Spaces	Int	11	
NumberParkingSpaces 		Number of Parking Spaces	Int	25	
NumberRemotes 		Num of Garage Openers/Remotes	Int	25	
NumberUnits 		Number of Units	Int	25	
Outdoors 		Outdoors	Character	236	Values
Parking 		Description of Parking Space	Character	60	Values
Patio 		Patio Description	Character	214	Values
PicCount 		Number of Pictures	Int	25	
Pool 		Pool Characteristics	Character	272	Values
PropertyDescription 		Property Description	Character	1275	
PropertySubType 	TYPE	Type	Character	60	Values
Roofing 		Roof	Character	360	Values
Rooms 		Rooms	Character	365	Values
RVAccessDimensions 		RV Access Dimensions	Character	7	
SA_Board 		saprimaryboard	Character	1	Values
SA_DreLicenseNumber 	SA Dre License Number	SA Dre License Number	Character	20	
SA_FirstName 		safname	Character	20	
SA_LastName 		salname	Character	20	
SA_PublicID 		Selling Agent Public ID	Character	30	
SaleType 	SaleType	SaleType	Character	255	Values
SchoolDistrict 	SCHOOLDISTRICT	School District	Character	60	Values
SchoolElementary 	ELEMENTARYSCHOOL	Elementary School	Character	20	
SchoolHigh 	HIGHSCHOOL	High School	Character	20	
SchoolJuniorHigh 	MIDDLESCHOOL	Junior High School	Character	20	
SellingPrice 		Selling Price	Int	0	
SeniorYN 		IsSeniorsHoursing	Boolean	5	
SO_Code 	SALEOFFICEBROKERID	Selling Office Code	Character	30	
SO_DreLicenseNumber 	SO_DreLicenseNumber	SO_DreLicenseNumber	Character	20	
SO_Name 	SALEOFFICENAME	Selling Office Name	Character	30	
Spa 		Spa Type	Character	60	Values
Sprinklers 		Lawn Sprinklers	Character	60	Values
SqFtSourceLot 		Lot Square Footage Source	Character	60	Values
SqFtSourceStructure 		Square Footage Source	Character	60	Values
SquareFootageStructure 		Square Footage	Int	0	
State 	STATE	State	Character	31	Values
Status 	LISTINGSTATUS	Status	Character	60	Values
Stories 		Story	Character	60	Values
StreetDirection 	STREETDIRPREFIX	Street Direction	Character	60	Values
StreetDirectionSuffix 	Street Direction Suffix	Street Direction Suffix	Character	255	Values
StreetName 	STREETNAME	Street Name	Character	30	
StreetNumber 	STREETNUMBER	Street Number	Int	25	
StreetNumberModifier 	StreetNumberModifier	StreetNumberModifier	Character	8	
StreetSuffix 	STREETSUFFIX	Street Suffix	Character	60	Values
StreetSuffixModifier 	StreetSuffixModifier	StreetSuffixModifier	Character	15	
Structure 		Structure	Character	402	Values
Style 		Style	Character	147	Values
ThomasGuide 		Thomas Guide Number	Character	7	
TractNumber 		Tract Number	Character	6	
UnitNumber 	UNITNUMBER	Unit Number	Character	6	
Utilities 		Water Source	Character	60	Values
View 		View	Character	267	Values
VirtualTour 		Virtual Tour	Character	250	
WellDepth 		Well Depth	Int	0	
WellGallonsPerMinute 		Gallons of Water Per Minute	Decimal	6	
WellPumpHorsepower 		Pump Motor Horsepower	Decimal	6	
WellReportYN 		WellReportYN	Boolean	5	
YearBuilt 	YEARBUILT	Year Built	Int	25	
YearBuiltSource 	YearBuiltSource	YearBuiltSource	Character	60	Values
ZipCode 	POSTALCODE	Zip Code	Character	10	
ZipCodePlus4 		ZipCodePlus4	Character	4	
Zone 		Zone	Character	40	



duntuk’s picture

StatusFileSize
new1.22 MB
duntuk’s picture

Hmm... I figured out a solution.

Changed 'Query Type' to 'Offset Not Supported (Query based on RETS Key)' which imported data sucessfully.

However, in 2.0 version i was able to keep this at: 'Default (Server must support offset. Relies on RETS Status Field.)' and it worked fine.

I'm guessing this probably a bug.

camidoo’s picture

Yea that is a bug. And you're 100% sure that your server supports offset, I'm assuming that it does as you said that you were able to import data before in the 2.x branch. I think i know what the problem might be.

camidoo’s picture

Assigned: Unassigned » camidoo

I'll take a look this morning.

duntuk’s picture

StatusFileSize
new348.66 KB
new214.79 KB

Thanks! I'm able to import data on 7.x-2.0-beta18 using offset setting for sure.

I'm running 2 separate installations on different sites.

I've included the 2.0 offset import... (yeah, there is a geocode error, but all other data imports fine)

e.g.

Saving item 57139467 [935.59 sec, 222.38 MB]                                                                                     [success]
Item 188 of 500 [935.59 sec, 222.36 MB]                                                                                             [notice]
Failed to Geocode: 1675 Maywood Avenue, Upland, California 91784 [935.59 sec, 222.38 MB]                                         [error]
Saving item 57139478 [935.6 sec, 222.38 MB]                                                                                      [success]
Item 189 of 500 [935.6 sec, 222.37 MB]                                                                                              [notice]
Failed to Geocode: 23844 Via Barletta , Murrieta, California 92562 [935.6 sec, 222.38 MB]                                        [error]
Saving item 57139719 [935.62 sec, 222.38 MB]                                                                                     [success]
Item 190 of 500 [935.63 sec, 222.37 MB]                                                                                             [notice]
Failed to Geocode: 11353 Siesta Lane, Corona, California 92883 [935.63 sec, 222.38 MB]                                           [error]
Saving item 57139720 [935.64 sec, 222.39 MB]                                                                                     [success]
Item 191 of 500 [935.64 sec, 222.37 MB]                                                                                             [notice]
Failed to Geocode: 13539 Pala Avenue, Sylmar, California 91342 [935.64 sec, 222.39 MB]                                           [error]
Saving item 57139750 [935.65 sec, 222.39 MB]                                                                                     [success]
Item 192 of 500 [935.65 sec, 222.38 MB]                                                                                             [notice]
Failed to Geocode: 15747 Joseph Trail , Lower Lake, California 95457 [935.65 sec, 222.39 MB] 
camidoo’s picture

Still haven't had time to look at this yet, however it's on my list

camidoo’s picture

were you ever able to get past this?

twod’s picture

Status: Active » Needs review
StatusFileSize
new1.48 KB

I get these "Undefined index" warnings on one server but not the other. Haven't looked closer at it but I think it happens because the returned XML is parsed into the $rets_item object but empty XML elements/tags aren't added as empty/NULL values to the object.

Anyway, this patch fixes the warnings for me.

jday’s picture

feedback: I applied the patch manually to the beta3 version to see if it would keep empty fields from displaying, unfortunately, they are still displaying...

twod’s picture

Do the should-be-empty fields happen to be numeric? If so, #1534508: Change the default numeric value to NULL should have fixed that.
However... Since that change didn't get in until beta3, any listing not updated/re-imported after you upgraded to beta3 may still be using a 0 instead of NULL in the database. Unless you flush and re-import everything, those 0s won't be changed to NULLs until the hash of the RETS fields for that listing changes.

The patch here will most likely not affect whether a field is considered empty or not because it only affects whether a field becomes NULL or undefined, which should both be similar enough to be considered empty. (Might be wrong tho.)

camidoo’s picture

Status: Needs review » Closed (fixed)

committed

camidoo’s picture

Issue summary: View changes

added note about error message