Commit graph

181 commits

Author SHA1 Message Date
Tobias Diekershoff 1bf089e424 Merge pull request #3216 from annando/1701-performance
Better network handling, improved structure
2017-03-12 15:40:56 +01:00
Michael da8a48f89e Bugfix: the curl option CURLOPT_SSL_VERIFYHOST is problematic 2017-02-25 13:59:24 +00:00
Michael f9f27c2f5f Merge remote-tracking branch 'upstream/develop' into 1701-performance 2017-02-23 06:55:35 +00:00
Hypolite Petovan 58a444b430 Add original_url() param documentation 2017-02-18 20:39:16 -05:00
Hypolite Petovan 2c959b925d Add param documentation 2017-02-18 20:35:31 -05:00
Hypolite Petovan 432587464c Fix Diaspora link attachment probe
- Move analytics param stripping out of original_url
- Remove HEAD curl request in ParseUrl::getSiteInfo
- Replace original_url with strip_tracking_query_params in
ParseUrl::getSiteInfo to prevent massive curl fest in border cases
2017-02-17 22:35:46 -05:00
Hypolite Petovan 99cfae63d7 Clean trailing whitespaces 2017-02-17 22:32:33 -05:00
Michael 7324a661f1 Optimized timeouts during network operations 2017-02-01 00:15:37 +00:00
Michael e67133ef56 Merge remote-tracking branch 'upstream/develop' into 1701-curl-range
Conflicts:
	mod/oexchange.php
	mod/uimport.php
2017-01-12 07:25:44 +00:00
Michael a2debaa68a Download limit for fetching data via "z_fetch_url" 2017-01-09 23:10:32 +00:00
Roland Häder f8a213e23b
*much* more usage of App::get_baseurl() instead of $a->get_baseurl() (coding convention applied)
Signed-off-by: Roland Häder <roland@mxchange.org>
2016-12-23 19:49:36 +01:00
Roland Häder be35349495 added more curyl braces + spaces between "if" and brace
Signed-off-by: Roland Häder <roland@mxchange.org>
2016-12-20 10:44:27 +01:00
Roland Häder 884f44ce94 *much* more usage of App::get_baseurl() instead of $a->get_baseurl() (coding convention applied)
Signed-off-by: Roland Häder <roland@mxchange.org>
2016-12-19 14:27:16 +01:00
Roland Häder 467cdfeb34 3 slashes for Doxygen, 2 are enough for PHP ... :-(
Signed-off-by: Roland Häder <roland@mxchange.org>
2016-12-14 17:28:45 +01:00
Roland Häder 23d4db5149 Continued with #3010:
- removed added debug messages (maybe they will come back in accordance to
  code-style guidelines?)
- converted $a->get_baseurl() back to App::get_baseurl()
- reverted back other code I have touched/merged as this was double:
  (substr($url, 0, 4) !== 'http') on $url = 'https://bla'; will be FALSE
  (means found http at start), too.

Signed-off-by: Roland Häder <roland@mxchange.org>
2016-12-14 16:59:35 +01:00
Roland Häder 63610c7b73 Merge branch 'rewrites/dbm_is_result' of github.com:Quix0r/friendica into rewrites/dbm_is_result
Signed-off-by: Roland Häder <roland@mxchange.org>

Conflicts:
	include/Photo.php
	include/api.php
	include/cache.php
	include/dbstructure.php
	include/enotify.php
	include/items.php
	include/onepoll.php
	include/queue_fn.php
	include/session.php
	include/text.php
	mod/contacts.php
	mod/dfrn_request.php
	mod/display.php
	mod/events.php
	mod/item.php
	mod/message.php
	mod/msearch.php
	mod/photos.php
	mod/poco.php
	mod/profile.php
	mod/proxy.php
	mod/videos.php
	update.php
	view/theme/diabook/theme.php
	view/theme/vier/theme.php
2016-12-13 11:00:44 +01:00
Michael 5ef688f1d2 Case correction 2016-11-14 05:44:18 +00:00
Michael 3de7b6d6ce Several shorteners doesn't work anymore 2016-11-13 19:19:35 +00:00
Roland Häder c825cc8d0d
This merge brings back dbm::is_result() where I could find it.
Merge branch 'develop' of github.com:friendica/friendica into rhaeder-develop

Signed-off-by: Roland Haeder <roland@mxchange.org>
2016-09-18 23:21:18 +02:00
Michael Vogel f27bc00464 Remove obsolote functions 2016-07-09 20:09:09 +02:00
Michael Vogel 4e5d335492 Removed the legacy function calls, removed unused functions 2016-07-04 08:05:30 +02:00
Roland Häder 432f96b646
And even more logging.
Signed-off-by: Roland Haeder <roland@mxchange.org>
2016-05-01 15:07:06 +02:00
Roland Häder f18e493e32
More logging!
Signed-off-by: Roland Haeder <roland@mxchange.org>
2016-05-01 15:01:19 +02:00
Roland Häder f88345cbea
HTTPS support in validate_url()
Signed-off-by: Roland Haeder <roland@mxchange.org>
2016-05-01 14:58:41 +02:00
Roland Häder 738e22d3eb
Some fixes:
- Letting $profile_link and $hcard always been overwritten with newly found
  entries has caused DFRN network detectio to fail and "connect" is no longer
  working. This fix makes sure that these both variables are only set if they
  are still empty.
- added more debug lines to see which URL was it before and after webfinger "detection"

Signed-off-by: Roland Haeder <roland@mxchange.org>
2016-05-01 14:52:48 +02:00
rabuzarus 95f129d9b6
xml: rename convert_element_to_array to element_to_array 2016-05-01 13:48:52 +02:00
rabuzarus 89f7a91907
move xml related function to the xml class + some doxygen docu 2016-05-01 13:48:52 +02:00
rabuzarus db581c08d0
rework autocomplete: some polishing 2016-05-01 13:47:24 +02:00
Michael Vogel 79fb05ff80
Removed moved function 2016-05-01 13:46:08 +02:00
rabuzarus 12479fb478
rework autocomplete: Access list of smilies over JSON 2016-05-01 13:46:08 +02:00
rabuzarus 9b3931e48a xml: rename convert_element_to_array to element_to_array 2016-04-27 17:04:46 +02:00
rabuzarus dbc39bea74 move xml related function to the xml class + some doxygen docu 2016-04-27 16:31:03 +02:00
rabuzarus ce1faf17d1 rework autocomplete: some polishing 2016-04-14 17:05:19 +02:00
rabuzarus 01b02dbcaa Merge develop into 1404_reworked_autocomplete
Conflicts:
	include/text.php
	view/templates/head.tpl
	view/theme/duepuntozero/style.css
	view/theme/vier/style.css
2016-04-14 16:23:51 +02:00
Michael Vogel 8027854886 Removed moved function 2016-03-15 07:18:11 +01:00
rabuzarus 5f7c5e6ab6 rework autocomplete: Access list of smilies over JSON 2016-02-01 17:43:12 +01:00
Matthew Exon 4ff8c9e3a1 Document new redirect_url return value 2016-01-25 11:38:36 +01:00
Matthew Exon 930da0aa44 Return redirect_url as part of the return array from z_fetch_url instead of via global state 2016-01-25 11:05:17 +01:00
Matthew Exon 3ada0093f9 Note the final URL used to retrieve content after all redirects 2016-01-25 07:41:56 +01:00
Matthew Exon 677903b316 Report errors during cURL fetch 2016-01-25 02:56:47 +01:00
Michael Vogel 5ec2c7e965 When a server isn't reachable there were timeout problems with fetching image data 2016-01-18 15:38:38 +01:00
rabuzarus 3df5c17124 reformat todo according to doxygen style 2015-12-25 23:17:34 +01:00
Michael Vogel 6a15dded0e Bugfix: The timeout value of z_fetch_url couldn't be set manually 2015-11-03 00:48:49 +01:00
Michael Vogel 2840bf2293 New option to permit crawlers 2015-10-03 23:16:40 +02:00
Michael Vogel 2038e65a57 Bugfix: The function "z_fetch_url" created warnings under certain circumstances. 2015-05-16 22:38:41 +02:00
fabrixxm 6258a21434 map bbcode tag from Red 2015-05-15 20:41:10 +02:00
Michael Vogel dbe8275ae5 Added some more timestamp measuring 2015-03-07 23:14:26 +01:00
Michael Vogel 7c4a1a059d New fields for the term table, improved query for the tag search. Changed the cache handling for rendered bbcode. 2015-03-07 21:24:39 +01:00
Michael Vogel 9e55dbf1db There is already a setting for that ... 2015-03-02 00:32:46 +01:00
Michael Vogel 5cb6d6e537 We have to disable the SSL checks, otherwise the communication will fail for servers with self-signed certificates 2015-03-01 10:53:49 +01:00
Fabrixxm caa21e6021 Merge branch 'master' into develop 2015-02-24 09:09:02 +01:00
Johannes Schwab 1cd1f7587c fix validate_email 2015-02-23 12:20:05 +01:00
Matthew Exon 2bb8b77123 Close cURL handle before recursing, so that cookie file is flushed 2015-02-11 03:42:07 +00:00
Matthew Exon d12b929c96 Accept lowercase location header 2015-02-10 03:14:08 +00:00
Michael Vogel aa6275356a The function to fetch the original url (without shortener) failed if a linked file was too large. 2015-01-04 15:31:47 +01:00
Michael Vogel 0c60eb18f7 Some more typical tracking parameter is removed 2014-12-01 22:52:51 +01:00
fabrixxm 6db73c0b6d fix update routine to support update from 3.2
disable PDO support
2014-09-07 17:28:38 +02:00
Michael Vogel 7af5c3c560 The useragent string doesn't contains of the words "Mozilla" anymore - but the name "Friendica" and the base url 2014-09-07 10:47:37 +02:00
Michael Vogel 07bc8ca166 fetch_url: Logging the data only when at the "data" loglevel 2014-08-07 08:22:32 +02:00
Michael Vogel e974e586cc Unified user agent string when fetching data. 2014-08-07 07:58:20 +02:00
Michael Vogel 6f5bc8d9a7 Setting the user agent to "Mozilla/5.0 (compatible; Friendica)" 2014-07-10 23:13:07 +02:00
Michael Vogel 6a28ae8b52 Just some better handling of tracker variables (url encoding) 2014-05-30 01:02:32 +02:00
Michael Vogel 135a403cec Added some tracking variables from facebook in the function to clean up URLs. 2014-05-29 11:43:37 +02:00
Michael Vogel 5ae9c5e538 Added some logging 2014-05-24 11:01:33 +02:00
Michael Vogel 6b461a74cb Cleaning up tracking parameters in urls. 2014-05-18 20:32:06 +02:00
Michael Vogel 426e0045ee The function "short_url" is used in several addons, so it deserves a central place. 2014-05-03 12:07:34 +02:00
Michael Vogel a64948b7f7 New function "original_url" (moved from an addon) that follows all redirects to the original url. 2014-04-04 10:55:30 +02:00
Michael Vogel 8c1d31decc Showing "view full size" is now configurable 2014-02-11 23:44:45 +01:00
fabrixxm ad5a47aecf add fetaure switch for acl-automention 2014-01-03 14:40:43 +01:00
Mats Sjöberg 19de78e511 Fixed subscription problem with new GNU Social. 2013-11-21 19:41:06 +01:00
Michael Vogel 759015e5fd Removed many "limit 1" at update and delete queries since they were superflous und generated massiv logfile entries 2013-11-02 16:13:11 +01:00
Matthew Exon 11163c6fb3 Allow calls to fetch_url to specify a cookiejar file, for sites that refuse to deliver content without setting a cookie 2013-08-05 10:59:22 +08:00
Matthew Exon 6a38705567 Replicate redirect URL building logic for post requests 2013-07-29 22:34:09 +08:00
Matthew Exon eb9bd03ef8 If the header returns an absolute path as a redirect, don't simply append the path to the old URL, remove the old absolute path first.
For example, original URL:

http://www.theregister.co.uk/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/print.html

Redirect from header:

/Print/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/

Incorrect result:

http://www.theregister.co.uk/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/print.html/Print/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/

Correct result after this patch:

http://www.theregister.co.uk/Print/2013/07/28/birmingham_uni_car_cracker_muzzled_by_lords/
2013-07-29 10:15:39 +08:00
friendica 886c3d1472 bad checkin 2013-05-02 22:19:16 -07:00
Matthew Exon 5b158dbcfb Make the content type available after calling fetch_url 2013-04-01 12:36:43 +08:00
Tobias Diekershoff e7c705c9a1 Merge pull request #654 from mexon/filter_var
Use filter_var instead of parse_url to validate URLs, since the latter r...
2013-03-26 11:31:11 -07:00
Matthew Exon c7cfc6b170 Use filter_var instead of parse_url to validate URLs, since the latter returns true even for invalid URLs, in particular the empty string 2013-03-21 16:27:12 +08:00
Matthew Exon c53bdfb163 Restore redirect functionality removed by change 452ff19d 2013-03-14 14:09:42 +08:00
friendica 280b72b9e6 Merge https://github.com/friendica/friendica into pull 2013-02-21 16:43:01 -08:00
friendica 615b060352 rev update 2013-02-21 16:42:34 -08:00
Michael Vogel fdee002f73 Several improvements for performance measuring 2013-01-26 18:35:39 +01:00
Michael - piratica.eu 2c93067682 Performance: Adding several variables for performance measuring. 2013-01-26 16:46:43 +01:00
Michael - piratica.eu 7ab335c9d6 Diaspora: Auto-adding users as friend who already sharing with us 2013-01-20 14:08:28 +01:00
Michael Vogel 452ff19d73 Redirects sometimes only seems to send the hostname and no path. "fetch_url" respects this. 2013-01-13 18:51:48 +01:00
Michael Vogel 5501dd0759 Searching for tags has been enabled in all places. The tag detection has been improved for imports with missing line feeds. 2013-01-13 17:13:01 +01:00
Michael Vogel a8aa992102 Vier: Design issues with preview fixed 2012-12-20 04:19:02 +01:00
Michael Vogel 46c5a97a20 New caching system for photos 2012-12-14 22:47:30 +01:00
Michael Vogel e36c30d285 Vier: Margin problems with code fixed. 2012-11-22 15:06:15 +01:00
Michael Vogel f17377e6de New cache system with subdirectories 2012-11-17 00:50:39 +01:00
friendica 621e737826 fix angle char storage and transmission, rev update 2012-08-03 18:33:11 -07:00
Michael Vogel fc27edb6d7 network: Caching of images 2012-07-26 00:23:25 +02:00
Michael Vogel 6456c2484e scale_external_images: Fix: Hadn't looked for pictures in the format [img=XxY] 2012-07-25 23:40:23 +02:00
Michael Vogel bfca7f47ef Merge remote branch 'upstream/master'
Conflicts:
	include/bb2diaspora.php
2012-07-13 23:39:51 +02:00
Michael Vogel 47c48aa438 DBA: Possibility to log the database speed 2012-07-08 21:27:20 +02:00
Michael Vogel e9d7f85297 network: Small changes 2012-07-07 20:40:15 +02:00
friendica dd71f9446b config to disable email validation 2012-07-01 18:56:00 -07:00
Zach Prezkuta 90f8454190 allow linking to Diaspora's scaled-down images 2012-06-23 11:02:15 -06:00
Fabio Comuni 7cc8c369cb Add support for PNG images with alpha 2012-06-07 17:42:13 +02:00
friendica 4f0e26759c bring in the *much better* xml parser from the original zot branch 2012-05-16 21:29:57 -07:00