Commit graph

89 commits

Author SHA1 Message Date
Michael Vogel 6a15dded0e Bugfix: The timeout value of z_fetch_url couldn't be set manually 2015-11-03 00:48:49 +01:00
Michael Vogel 2840bf2293 New option to permit crawlers 2015-10-03 23:16:40 +02:00
Michael Vogel 2038e65a57 Bugfix: The function "z_fetch_url" created warnings under certain circumstances. 2015-05-16 22:38:41 +02:00
fabrixxm 6258a21434 map bbcode tag from Red 2015-05-15 20:41:10 +02:00
Michael Vogel dbe8275ae5 Added some more timestamp measuring 2015-03-07 23:14:26 +01:00
Michael Vogel 7c4a1a059d New fields for the term table, improved query for the tag search. Changed the cache handling for rendered bbcode. 2015-03-07 21:24:39 +01:00
Michael Vogel 9e55dbf1db There is already a setting for that ... 2015-03-02 00:32:46 +01:00
Michael Vogel 5cb6d6e537 We have to disable the SSL checks, otherwise the communication will fail for servers with self-signed certificates 2015-03-01 10:53:49 +01:00
Fabrixxm caa21e6021 Merge branch 'master' into develop 2015-02-24 09:09:02 +01:00
Johannes Schwab 1cd1f7587c fix validate_email 2015-02-23 12:20:05 +01:00
Matthew Exon 2bb8b77123 Close cURL handle before recursing, so that cookie file is flushed 2015-02-11 03:42:07 +00:00
Matthew Exon d12b929c96 Accept lowercase location header 2015-02-10 03:14:08 +00:00
Michael Vogel aa6275356a The function to fetch the original url (without shortener) failed if a linked file was too large. 2015-01-04 15:31:47 +01:00
Michael Vogel 0c60eb18f7 Some more typical tracking parameter is removed 2014-12-01 22:52:51 +01:00
fabrixxm 6db73c0b6d fix update routine to support update from 3.2
disable PDO support
2014-09-07 17:28:38 +02:00
Michael Vogel 7af5c3c560 The useragent string doesn't contains of the words "Mozilla" anymore - but the name "Friendica" and the base url 2014-09-07 10:47:37 +02:00
Michael Vogel 07bc8ca166 fetch_url: Logging the data only when at the "data" loglevel 2014-08-07 08:22:32 +02:00
Michael Vogel e974e586cc Unified user agent string when fetching data. 2014-08-07 07:58:20 +02:00
Michael Vogel 6f5bc8d9a7 Setting the user agent to "Mozilla/5.0 (compatible; Friendica)" 2014-07-10 23:13:07 +02:00
Michael Vogel 6a28ae8b52 Just some better handling of tracker variables (url encoding) 2014-05-30 01:02:32 +02:00
Michael Vogel 135a403cec Added some tracking variables from facebook in the function to clean up URLs. 2014-05-29 11:43:37 +02:00
Michael Vogel 5ae9c5e538 Added some logging 2014-05-24 11:01:33 +02:00
Michael Vogel 6b461a74cb Cleaning up tracking parameters in urls. 2014-05-18 20:32:06 +02:00
Michael Vogel 426e0045ee The function "short_url" is used in several addons, so it deserves a central place. 2014-05-03 12:07:34 +02:00
Michael Vogel a64948b7f7 New function "original_url" (moved from an addon) that follows all redirects to the original url. 2014-04-04 10:55:30 +02:00
Michael Vogel 8c1d31decc Showing "view full size" is now configurable 2014-02-11 23:44:45 +01:00
fabrixxm ad5a47aecf add fetaure switch for acl-automention 2014-01-03 14:40:43 +01:00
Mats Sjöberg 19de78e511 Fixed subscription problem with new GNU Social. 2013-11-21 19:41:06 +01:00
Michael Vogel 759015e5fd Removed many "limit 1" at update and delete queries since they were superflous und generated massiv logfile entries 2013-11-02 16:13:11 +01:00
Matthew Exon 11163c6fb3 Allow calls to fetch_url to specify a cookiejar file, for sites that refuse to deliver content without setting a cookie 2013-08-05 10:59:22 +08:00
Matthew Exon 6a38705567 Replicate redirect URL building logic for post requests 2013-07-29 22:34:09 +08:00
Matthew Exon eb9bd03ef8 If the header returns an absolute path as a redirect, don't simply append the path to the old URL, remove the old absolute path first.
For example, original URL:

http://www.theregister.co.uk/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/print.html

Redirect from header:

/Print/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/

Incorrect result:

http://www.theregister.co.uk/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/print.html/Print/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/

Correct result after this patch:

http://www.theregister.co.uk/Print/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/
2013-07-29 10:15:39 +08:00
friendica 886c3d1472 bad checkin 2013-05-02 22:19:16 -07:00
Matthew Exon 5b158dbcfb Make the content type available after calling fetch_url 2013-04-01 12:36:43 +08:00
Tobias Diekershoff e7c705c9a1 Merge pull request #654 from mexon/filter_var
Use filter_var instead of parse_url to validate URLs, since the latter r...
2013-03-26 11:31:11 -07:00
Matthew Exon c7cfc6b170 Use filter_var instead of parse_url to validate URLs, since the latter returns true even for invalid URLs, in particular the empty string 2013-03-21 16:27:12 +08:00
Matthew Exon c53bdfb163 Restore redirect functionality removed by change 452ff19d 2013-03-14 14:09:42 +08:00
friendica 280b72b9e6 Merge https://github.com/friendica/friendica into pull 2013-02-21 16:43:01 -08:00
friendica 615b060352 rev update 2013-02-21 16:42:34 -08:00
Michael Vogel fdee002f73 Several improvements for performance measuring 2013-01-26 18:35:39 +01:00
Michael - piratica.eu 2c93067682 Performance: Adding several variables for performance measuring. 2013-01-26 16:46:43 +01:00
Michael - piratica.eu 7ab335c9d6 Diaspora: Auto-adding users as friend who already sharing with us 2013-01-20 14:08:28 +01:00
Michael Vogel 452ff19d73 Redirects sometimes only seems to send the hostname and no path. "fetch_url" respects this. 2013-01-13 18:51:48 +01:00
Michael Vogel 5501dd0759 Searching for tags has been enabled in all places. The tag detection has been improved for imports with missing line feeds. 2013-01-13 17:13:01 +01:00
Michael Vogel a8aa992102 Vier: Design issues with preview fixed 2012-12-20 04:19:02 +01:00
Michael Vogel 46c5a97a20 New caching system for photos 2012-12-14 22:47:30 +01:00
Michael Vogel e36c30d285 Vier: Margin problems with code fixed. 2012-11-22 15:06:15 +01:00
Michael Vogel f17377e6de New cache system with subdirectories 2012-11-17 00:50:39 +01:00
friendica 621e737826 fix angle char storage and transmission, rev update 2012-08-03 18:33:11 -07:00
Michael Vogel fc27edb6d7 network: Caching of images 2012-07-26 00:23:25 +02:00